Departement Elektrotechniek ESAT-SISTA/TR Dynamical System Prediction: a Lie algebraic approach for a novel. neural architecture 1

Size: px
Start display at page:

Download "Departement Elektrotechniek ESAT-SISTA/TR Dynamical System Prediction: a Lie algebraic approach for a novel. neural architecture 1"

Transcription

1 Katholieke Universiteit Leuven Departement Elektrotechniek ESAT-SISTA/TR Dynamical System Prediction: a Lie algebraic approach for a novel neural architecture 1 Yves Moreau and Joos Vandewalle 2 May 1993 Accepted for publication in Annals of Mathematics and Articial Intelligence 1 This report is available by anonymous ftp from ftp.esat.kuleuven.ac.be in the directory pub/sista/moreau/reports/manna95 tr95-47.ps.z 2 ESAT - Katholieke Universiteit Leuven, Kardinaal Mercierlaan 94, 3001 Leuven (Heverlee), Belgium, tel 32/1/ , fax 32/1/ , moreau@esat.kuleuven.ac.be.

2 Abstract In this paper we study the problem of the prediction of autonomous continuous-time dynamical systems from discrete-time measurements of the state variables. From the fact that the system is inherently continuous in time but that the measurements are made at discrete time-intervals, we show that the predictor of such a system needs to be an invertible map from the state-space into itself. The problem then becomes one of how to approximate invertible maps. We show that standard approximation schemes do not guarantee the property of invertibility. We therefore propose a new approximation scheme which is based on the composition of invertible basis functions and which preserves invertibility. This approach can be cast in a Lie algebraic framework where the approximation is based on the use of the Baker-Campbell-Hausdor formula. The method can be implemented in a neural-like form and we illustrate it by an example. We also present a more general implementation which we call "MLP in dynamics space". Keywords: dynamical system prediction, Lie algebras, Baker-Campbell-Hausdor formula

3 1 Introduction We will study here the problem of the prediction of nonlinear dynamical systems from a set of sampled measurements of the state of the system. The nonlinear systems which we will consider are those nonlinear autonomous dynamical systems whose dynamics is continuous in time and therefore governed by a multidimensional ordinary dierential equation. However, although the dynamics of the system is continuous in time, the samples that we will use for prediction are the state variables measured at discrete time-intervals only. So, we are led to postulate a discretetime form for our prediction: ^x(k + 1) = F (x(k)). Using the fact that the underlying dynamics of the system is time-continuous, we will show that the function F used for prediction should be invertible. The problem then becomes that of building an approximation scheme specically designed for invertible maps. The main drawback of standard approximation schemes is that they are based on summing basis functions to build an approximation. However, this operation does not preserve invertibility. Using the operation of composition instead of addition, we are able to preserve invertibility. We will thus build approximation schemes based on composition of invertible basis functions. Those compositions can be presented in the framework of Lie algebra theory. Starting from a method from numerical analysis we will be able to derive a new approach to the prediction of nonlinear systems. This new method can easily be presented in a neural-like form. This work is organized as follows. In the rst section we will recall the basic concept of dynamical system theory and we will show that a predictor of a dynamical system should be invertible. In the second section we will present the elements of Lie algebra theory and how they can be applied, using the Baker-Campbell-Hausdor formula, to build approximation schemes based on compositions. In the third section we will present the network implementation of our model which we will illustrate by an example. We will also present a general model for approximation which we call "MLP in dynamics space". Finally, we will conclude. 1.1 Dynamical systems A dynamical system is a rule for the evolution over time of a set of variables. This set of variables is called the state of the system. Following the general dierential geometric denition of dynamical systems [2], the state will be identied here with some point on a manifold M. We will consider here only those systems for which the state evolves according to some ordinary dierential equation _x(t) = f(x(t)). The function f is the vector eld of the system and it associates to each point of the manifold M an element of the tangent space T x M. A solution to the ODE is a trajectory of the system. Such a system can thus be written as: M a manifold x(t) 2 M for all t f : M! T x M _x(t) = f(x(t)) (1) 1

4 1.2 The initial value problem and the existence, uniqueness and smoothness of solutions The initial value problem (IVP) simply consists in nding the trajectory passing through a given initial state x 0 : ( _x(t) = f(x(t)) x(0) = x 0 (2) A sucient condition for the existence of a unique solution to the IVP is that the vector eld f be everywhere locally Lipschitz [4]. The solution might however be dened only for small enough positive and negative times as the trajectory might escape from the manifold after some time. If we require the vector eld to be C r (1 r 1), the solution x(t) will be C r+1 with respect to time t and C r with respect to a change in initial condition x 0. From here on we will always assume the vector eld to be at least locally Lipschitz and we will assume further smoothness wherever necessary. 1.3 The ow of a dynamical system For each initial condition x 0, we can solve the initial value problem. We then decide to collect all those solutions in one function (x 0 ; t) which gives the solution of the ODE as a function of the initial condition: 8 > < >: _x = f(x(t)) ) x(t) = (x 0 ; t) (3) x(0) = x 0 is called the ow of the dynamical system. The ow is a remarkable object which possesses the group property. We refer the reader to [4] for a general presentation of the group theoretic approach to dynamical systems. If we assume, for simplicity, that the ow is dened for all times, the group property writes as follows: : R M! M (x; t + s) = ((x; t); s) 8 t; s 2 R (4) As we will see, this gives to the ow the structure of a group. We rst dene the time-t map as: ' t (x 0 ) (x 0 ; t) (5) This map ' t associates to any initial condition its image under the ow of the dierential equation after a given time t. So, we have on one hand a trajectory which is a way of looking at the evolution of a given initial condition for all possible times. And we have on the other hand a time-t map which is a way to look at the evolution of all possible initial conditions but only for one given time.the map ' t is a map from M into itself. Using the group property of the ow, we can deduce the properties of ': ' t ' s (x 0 ) = ((x 0 ; s); t) = (x 0 ; t + s) = ' t+s (x 0 ) ' 0 = I ( ' 0 (x 0 ) = (x 0 ; 0) = x 0 = I(x 0 ) ' t '?t (x0) = (x 0 ; 0) = x 0 = I(x 0 ) ) '?t = (' t )?1 composition identity inverse () 2

5 Moreover, if f is C r then ' is C r. Hence, ' is a smooth invertible map with a smooth inverse, that is, ' is a dieomorphism. Furthermore, the properties of ' show that f' t g is a group. We therefore say that a ow (x; t) is a one-parameter group of dieomorphisms ' t. 1.4 Dynamical system prediction Even if we assume that the underlying dynamics of the system is governed by an ODE, the system is generally observed only at discrete time-intervals. So, if we choose a sampling period t, we can assume a functional relationship between the current state and the next sample. The system becomes of the form: x((k + 1):t) = F (x(k:t)); x(k) 2 M (7) And, from what we have shown before, we can see that: Hence, F is a dieomorphism. F (:) = ' t (:) (8) The following gure (Fig. 1) summarizes those observations. On the left we consider a square grid of initial conditions. We then let each point of that grid follow its trajectory for a timeinterval t. After this time, all the points of the grid have moved and the shape of the grid has changed. The dieomorphism ' t is that transformation from the manifold (here, R 2 ) into itself which maps the square grid on the left into the deformed grid on the right. One can immediately see that this transformation is smooth and invertible, hence a dieomorphism. ' t (x 0 ) = (x 0 ; t) Figure 1: Dieomorphic transformation of a square grid after one time-step. So, the question of predicting a continuous-time system observed at discrete time-intervals becomes one of how we can approximate those dieomorphisms which arise from the sampling of an ODE. The basic idea is that the set of all C r dieomorphisms is also a group. In a group, the natural operation is the composition of the elements of the group. While most standard approximation schemes are based on the summation of some basis functions, this operation does not preserve the property of being a dieomorphism. On the other hand, composition does preserve this property, so we propose to build an approximation scheme based on composing basis 3

6 functions instead of adding them. Schematically, one could caricature this by saying that the transformation in Fig. 1 can be approximated by a sequence of two successive transformations. The rst transformation could for example pinch the left and right sides of the square and then be followed by a second one which would be a clockwise rotation of about 15 degrees. 2 Lie algebra theory Lie algebra theory has a long history in physics, mostly in the areas of classical mechanics [1] and partial dierential equations [8]. It is also an essential part of nonlinear system theory [5]. Our approach to system prediction can be best cast in this framework. A Lie algebra is dened as follows: Denition.1 let V be a vector space; V is a Lie algebra; with Lie bracket [:; :] : V V! V i for all A; B; A 1 ; A 2 ; C 2 V : [A; B] =?[B; A] antisymmetry [a:a; B] = a:[a; B] [A 1 + A 2 ; B] = [A 1 ; B] + [A 2 ; B] bilinearity [[A; B]; C] + [[B; C]; A] + [[C; A]; B] = 0 Jacobi identity (9) In the case where the Lie algebra is the vector space of all C r vector elds, the time-t map ' t plays a very important role. If the vector eld is A, we dene the exponential map as follows: exp(t:a) ' t. This notation is an extension of the case where the vector eld is linear in space and where the solution is given by exponentiation of the matrix. The exponential is thus a mapping from the manifold into itself. And this map depends on the underlying vector eld A and changes over time. From here on, the product of exponentials will denote the composition of the maps. One can then dene the Lie bracket by: [A; B] exp(?s:b): exp(?t:a): exp(s:b): exp(t:a) t=s=0 The bracket [A; B] is called the commutator of the vector elds as it measures the degree of noncommutativity of the ows of the vector elds ( [A; B] = 0, exp(a): exp(b) = exp(b): exp(a)). In the case where the manifold is R n, the bracket can be further particularized to: nx [A; B] i = B i j? A i j i i=1 4

7 2.1 The Baker-Campbell-Hausdor formula The Baker-Campbell-Hausdor (BCH) formula gives an expansion for the product of two exponentials of elements of the Lie algebra, see []. exp(a): exp(b) = exp(a + B [A; B] ([A; [A; B]]? [B; [B; A]]) + [A; [B; [B; A]]] + : : :) (12) The problem of predicting a dynamical system with vector eld X then becomes that of building an approximation for exp(t:x) as we have that: x(k + 1) = exp(t:x)[x(k)] (13) This problem has recently been the focus of much attention in the eld of numerical analysis for the integration of Hamiltonian dierential equations [] [7]. Suppose that the vector eld X is of the form X = A + B where we can integrate A and B directly. Then one can use the BCH formula to produce a rst-order approximation to the exponential map: BCH : exp(t:x) = exp(t:a): exp(t:b) + o(t 2 ) (14) This is the essence of the method as it shows that one can approximate an exponential map (that is the mapping arising from the solution of an ODE) by composing simpler maps. By further manipulating the BCH formula, we can show that the following leapfrog scheme is second-order. Leapfrog : exp(t:x) = exp( t :A): exp(t:b): exp(t 2 2 :A) + o(t3 ) (15) Using this leapfrog scheme as a basis element for further leapfrog schemes. Yoshida [9] showed that it was in fact possible to produce an approximation to exp(t:x) up to any order. Order p : 9k; 9w 1 ; v 1 ; : : :; w k ; v k : exp(t:x) = exp(w 1 :t:a): exp(v 1 :t:b) : : : exp(w k :t:a): exp(v k :t:b) + o(t p+1 ) (1) Forest and Ruth [3] showed that approximations could be built for more than two vector elds. Combining the two we can state that it is possible to build an approximation to the solution of a linear combination of vector elds as a product of exponential maps. X = P m i=1 a i :A i ) 9w ij : exp(t:x) = Q k j=1 Q mi=1 exp(w ij :t:a i ) + o(t p+1 ) (17) 3 Network implementation Such an approximation scheme can be easily implemented as a network as can be seen in the following gure (Fig. 2). The problem of predicting the system can now simply be solved by minimizing the prediction error of the model by tuning the weights w ij. Moreover, gradients can easily be computed which allows for a gradient-descent, backpropagation-like implementation or any other gradient based optimization method. 5

8 ^x(k + 1) exp(w 22 A 2 ) exp(w 21 A 1 ) exp(w 12 A 2 ) exp(w 11 A 1 ) x(k) w 22 w 21 w 12 w 11 Figure 2: Implementation of the composition network. 3.1 Example We will now present an example to illustrate how this method can be applied. We will do this by looking at the Vanderpol oscillator. This system is described by the following second-order nonlinear dierential equation: x? :(1? x 2 ): _x + x = 0 (18) It can be directly written as a rst-order system in state-space form: ( _x = y _y =?x + :(1? x 2 ):y This system can be seen as a linear combination of two vector elds which can be solved analytically:!!! _x y 0 = +: _y?x (1? x 2 ):y (20) = A 1 + :A 2 One can then build a predictor according to the architecture just presented (Fig. 2), simply choosing an appropriate number of layers. One then nds the appropriate weights w ij by minimizing the error on the training set. One interesting property of this method is that we were able to use the a priori knowledge we had about the system. Such a feature is uncommon in the eld of system prediction using neural networks. This method allowed us to build a predictor for the Vanderpol oscillator in the case where the parameter was unknown. Using the same method we were able to build a predictor for the Lorentz attractor in the case where the parameters were also unknown. (19)

9 3.2 MLP in dynamics space If we decide to approximate the vector eld of our system by a multi-layer perceptron, one can derive a composition network to implement this. The form of the vector eld is: X(x) = nx j=1 ~c j :( ~ b j :x + b j0 ) = X j=1 where (x) = tanh(x) and ~c j ; ~ b j 2 R d+1 ; j = 1; :::; n. A ~c j; ~ b j j (x) (21) But the dierential equation _x = (x) can be solved explicitly in the one-dimensional case. We can use this to explicitly integrate the multidimensional system _x = A ~c;~ b (x) for any value of the parameters ~c j ; ~ b j. So, we can design a network of the following form: ^x(k + 1) = F (x(k)) = exp(w k :A ~c k; ~ b k ) : : : exp(w 1 :A ~c 1; ~ b 1 )(x(k)) (22) We call this an \MLP in dynamics space" as the MLP is implicitly used to parametrize the vector eld. Unfortunately, preliminary simulations have shown so far that it is very hard to nd the optimal coecients of such a model using only brute force gradient optimization. Further insight is needed in order to nd less computationally demanding methods for the training of such a network. 4 Conclusions We have studied the problem of predicting an autonomous continuous-time nonlinear dynamical system from discrete-time measurements of its state. We showed that because the predictor must have the form of an invertible map, it is interesting to have an approximation scheme which has this property. This can be achieved by using compositions instead of additions in the approximation. The theory of Lie algebras provided a framework to develop such a method, in a fashion similar to what is done for some numerical methods of integration of ordinary dierential equations. We showed that this resulted in a neural-like architecture which we illustrated by a simple example, and that it could be further extended to some form of "MLP in dynamics space". Further work will be primarily devoted to practical applications of this scheme, especially to the design of ecient training algorithms. Acknowledgments This research work was carried out at the ESAT laboratory of the Katholieke Universiteit Leuven, in the framework of a Concerted Action Project of the Flemish Community, entitled Model-based Information Processing Systems GOA-MIPS, and within the framework of the Belgian program on inter-university attraction poles (IUAP-17) initiated by the Belgian State, Prime Minister's Oce for Science, Technology and Culture, and within the framework of the European Commission Human Capital and Mobility SIMONET (System Identication and Modeling Network). The scientic responsibility rests with its authors. Yves Moreau is a research assistant of the N.F.W.O. (Belgian National Fund for Scientic Research). 7

10 References [1] V.I. Arnold, Mathematical methods of classical mechanics., Springer-Verlag, New York, [2] D.R.J.Chillingworth, Dierential topology with a view to applications., Pitman Publishing, London, [3] E. Forest, R. Ruth, Fourth-order symplectic integration, Physica D, 43 (1990) [4] M.C. Irwin, Smooth Dynamical Systems., Academic Press, New York, [5] A. Isidori, Nonlinear control theory, Springer-Verlag, New York, [] P.-V. Kosele, Calcul formel pour les methodes de Lie en mecanique hamiltonienne, Ph.D. thesis, Ecole Polytechnique, Paris, [7] R.I. McLachlan, On the numerical integration of ordinary dierential equations by symmetric composition methods, SIAM J. Sci. Comput., 1(1) (1995) [8] P.J Olver, Applications of Lie groups to dierential equations., Springer-Verlag, New York, [9] H. Yoshida, Construction of higher order symplectic integrators, Phys. Lett. A, 150 (1990)

Accelerate Implementation of Forwaring Control Laws using Composition Methos Yves Moreau an Roolphe Sepulchre June 1997 Abstract We use a metho of int

Accelerate Implementation of Forwaring Control Laws using Composition Methos Yves Moreau an Roolphe Sepulchre June 1997 Abstract We use a metho of int Katholieke Universiteit Leuven Departement Elektrotechniek ESAT-SISTA/TR 1997-11 Accelerate Implementation of Forwaring Control Laws using Composition Methos 1 Yves Moreau, Roolphe Sepulchre, Joos Vanewalle

More information

COMPOSITION METHODS FOR THE SIMULATION OF ARRAYS OF CHUA S CIRCUITS

COMPOSITION METHODS FOR THE SIMULATION OF ARRAYS OF CHUA S CIRCUITS International Journal of Bifurcation and Chaos, Vol. 9, No. 4 (1999) 723 733 c World Scientific Publishing Company COMPOSITION METHODS FOR THE SIMULATION OF ARRAYS OF CHUA S CIRCUITS YVES MOREAU and JOOS

More information

Optimal trac light control for a single intersection Bart De Schutter and Bart De Moor y ESAT-SISTA, KU Leuven, Kardinaal Mercierlaan 94, B-3 Leuven (

Optimal trac light control for a single intersection Bart De Schutter and Bart De Moor y ESAT-SISTA, KU Leuven, Kardinaal Mercierlaan 94, B-3 Leuven ( Katholieke Universiteit Leuven Departement Elektrotechniek ESAT-SISTA/TR 97- Optimal trac light control for a single intersection Bart De Schutter and Bart De Moor Proceedings of the 997 International

More information

x 104

x 104 Departement Elektrotechniek ESAT-SISTA/TR 98-3 Identication of the circulant modulated Poisson process: a time domain approach Katrien De Cock, Tony Van Gestel and Bart De Moor 2 April 998 Submitted for

More information

The Linear Dynamic Complementarity Problem is a special case of the Extended Linear Complementarity Problem B. De Schutter 1 B. De Moor ESAT-SISTA, K.

The Linear Dynamic Complementarity Problem is a special case of the Extended Linear Complementarity Problem B. De Schutter 1 B. De Moor ESAT-SISTA, K. Katholieke Universiteit Leuven Departement Elektrotechniek ESAT-SISTA/TR 9-1 The Linear Dynamic Complementarity Problem is a special case of the Extended Linear Complementarity Problem 1 Bart De Schutter

More information

On max-algebraic models for transportation networks

On max-algebraic models for transportation networks K.U.Leuven Department of Electrical Engineering (ESAT) SISTA Technical report 98-00 On max-algebraic models for transportation networks R. de Vries, B. De Schutter, and B. De Moor If you want to cite this

More information

Embedding Recurrent Neural Networks into Predator-Prey Models Yves Moreau, St phane Louies 2, Joos Vandewalle, L on renig 2 Katholieke Universiteit Le

Embedding Recurrent Neural Networks into Predator-Prey Models Yves Moreau, St phane Louies 2, Joos Vandewalle, L on renig 2 Katholieke Universiteit Le Katholieke Universiteit Leuven Departement Elektrotechniek ESAT-SISTA/TR 998-2 Embedding Recurrent Neural Networks into Predator-Prey Models Yves Moreau 2, Stephane Louies 3, Joos Vandewalle 2, Leon renig

More information

Departement Elektrotechniek ESAT-SISTA/TR 98- Stochastic System Identication for ATM Network Trac Models: a Time Domain Approach Katrien De Cock and Bart De Moor April 998 Accepted for publication in roceedings

More information

Elementary Approximation of Exponentials of. Frederic Jean & Pierre{Vincent Koseleff. Equipe Analyse Algebrique, Institut de Mathematiques

Elementary Approximation of Exponentials of. Frederic Jean & Pierre{Vincent Koseleff. Equipe Analyse Algebrique, Institut de Mathematiques Elementary Approximation of Exponentials of Lie Polynomials? Frederic Jean & Pierre{Vincent Koseleff Equipe Analyse Algebrique, Institut de Mathematiques Universite Paris 6, Case 8 4 place Jussieu, F-755

More information

Abstract. In this paper we study the performance of a symplectic numerical integrator

Abstract. In this paper we study the performance of a symplectic numerical integrator Geometric Numerical Integration Applied to The Elastic Pendulum at Higher Order Resonance J. M. Tuwankotta y G.R.W. Quispel E-mail: tuwankotta@math.uu.nl Email: R.Quispel@latrobe.edu.au Mathematisch Instituut,

More information

Master-Slave Synchronization using. Dynamic Output Feedback. Kardinaal Mercierlaan 94, B-3001 Leuven (Heverlee), Belgium

Master-Slave Synchronization using. Dynamic Output Feedback. Kardinaal Mercierlaan 94, B-3001 Leuven (Heverlee), Belgium Master-Slave Synchronization using Dynamic Output Feedback J.A.K. Suykens 1, P.F. Curran and L.O. Chua 1 Katholieke Universiteit Leuven, Dept. of Electr. Eng., ESAT-SISTA Kardinaal Mercierlaan 94, B-1

More information

28344 Bremen, Germany. hdullin or

28344 Bremen, Germany.   hdullin or Ecient Calculation of Actions H.R. Dullin A. Wittek Institut fur Theoretische Physik Universitat Bremen Postfach 330440 28344 Bremen, Germany Phone: +49(0)421-218-2341 Fax: +49(0)421-218-4869 Email: hdullin

More information

THEODORE VORONOV DIFFERENTIABLE MANIFOLDS. Fall Last updated: November 26, (Under construction.)

THEODORE VORONOV DIFFERENTIABLE MANIFOLDS. Fall Last updated: November 26, (Under construction.) 4 Vector fields Last updated: November 26, 2009. (Under construction.) 4.1 Tangent vectors as derivations After we have introduced topological notions, we can come back to analysis on manifolds. Let M

More information

ZIG-ZAG DYNAMICAL SYSTEMS AND THE BAKER-CAMPBELL-HAUSDORFF FORMULA. Alois Klc Pavel Pokorny Jan Rehacek. (Communicated by Milan Medved')

ZIG-ZAG DYNAMICAL SYSTEMS AND THE BAKER-CAMPBELL-HAUSDORFF FORMULA. Alois Klc Pavel Pokorny Jan Rehacek. (Communicated by Milan Medved') Math. Slovaca 5 () No. 1 nn{nn aoc Mathematical Institute Slovak Academy of Sciences ZIG-ZAG DYNAMICAL SYSTEMS AND THE BAKER-CAMPBELL-HAUSDORFF FORMULA Alois Klc Pavel Pokorny Jan Rehacek (Communicated

More information

The Erwin Schrodinger International Boltzmanngasse 9. Institute for Mathematical Physics A-1090 Wien, Austria

The Erwin Schrodinger International Boltzmanngasse 9. Institute for Mathematical Physics A-1090 Wien, Austria ESI The Erwin Schrodinger International Boltzmanngasse 9 Institute for Mathematical Physics A-1090 Wien, Austria Noncommutative Contact Algebras Hideki Omori Yoshiaki Maeda Naoya Miyazaki Akira Yoshioka

More information

Optimal traffic light control for a single intersection

Optimal traffic light control for a single intersection KULeuven Department of Electrical Engineering (ESAT) SISTA Technical report 97- Optimal traffic light control for a single intersection B De Schutter and B De Moor If you want to cite this report, please

More information

REMARKS ON THE TIME-OPTIMAL CONTROL OF A CLASS OF HAMILTONIAN SYSTEMS. Eduardo D. Sontag. SYCON - Rutgers Center for Systems and Control

REMARKS ON THE TIME-OPTIMAL CONTROL OF A CLASS OF HAMILTONIAN SYSTEMS. Eduardo D. Sontag. SYCON - Rutgers Center for Systems and Control REMARKS ON THE TIME-OPTIMAL CONTROL OF A CLASS OF HAMILTONIAN SYSTEMS Eduardo D. Sontag SYCON - Rutgers Center for Systems and Control Department of Mathematics, Rutgers University, New Brunswick, NJ 08903

More information

SVD-based optimal ltering with applications to noise reduction in speech signals Simon Doclo ESAT - SISTA, Katholieke Universiteit Leuven Kardinaal Me

SVD-based optimal ltering with applications to noise reduction in speech signals Simon Doclo ESAT - SISTA, Katholieke Universiteit Leuven Kardinaal Me Departement Elektrotechniek ESAT-SISTA/TR 999- SVD-based Optimal Filtering with Applications to Noise Reduction in Speech Signals Simon Doclo, Marc Moonen April, 999 Internal report This report is available

More information

Generalized Shifted Inverse Iterations on Grassmann Manifolds 1

Generalized Shifted Inverse Iterations on Grassmann Manifolds 1 Proceedings of the Sixteenth International Symposium on Mathematical Networks and Systems (MTNS 2004), Leuven, Belgium Generalized Shifted Inverse Iterations on Grassmann Manifolds 1 J. Jordan α, P.-A.

More information

THE RICCI BRACKET FLOW FOR SOLVABLE LIE GROUPS

THE RICCI BRACKET FLOW FOR SOLVABLE LIE GROUPS THE RICCI BRACKET FLOW FOR SOLVABLE LIE GROUPS KEVIN KOZLOWSKI AND BRENNAN VINCENT Abstract. The Ricci bracket flow is a geometric evolution on Lie algebras which is related to the Ricci flow on the corresponding

More information

On Mean Curvature Diusion in Nonlinear Image Filtering. Adel I. El-Fallah and Gary E. Ford. University of California, Davis. Davis, CA

On Mean Curvature Diusion in Nonlinear Image Filtering. Adel I. El-Fallah and Gary E. Ford. University of California, Davis. Davis, CA On Mean Curvature Diusion in Nonlinear Image Filtering Adel I. El-Fallah and Gary E. Ford CIPIC, Center for Image Processing and Integrated Computing University of California, Davis Davis, CA 95616 Abstract

More information

below, kernel PCA Eigenvectors, and linear combinations thereof. For the cases where the pre-image does exist, we can provide a means of constructing

below, kernel PCA Eigenvectors, and linear combinations thereof. For the cases where the pre-image does exist, we can provide a means of constructing Kernel PCA Pattern Reconstruction via Approximate Pre-Images Bernhard Scholkopf, Sebastian Mika, Alex Smola, Gunnar Ratsch, & Klaus-Robert Muller GMD FIRST, Rudower Chaussee 5, 12489 Berlin, Germany fbs,

More information

Definition 5.1. A vector field v on a manifold M is map M T M such that for all x M, v(x) T x M.

Definition 5.1. A vector field v on a manifold M is map M T M such that for all x M, v(x) T x M. 5 Vector fields Last updated: March 12, 2012. 5.1 Definition and general properties We first need to define what a vector field is. Definition 5.1. A vector field v on a manifold M is map M T M such that

More information

NL q theory: checking and imposing stability of recurrent neural networks for nonlinear modelling J.A.K. Suykens, J. Vandewalle, B. De Moor Katholieke

NL q theory: checking and imposing stability of recurrent neural networks for nonlinear modelling J.A.K. Suykens, J. Vandewalle, B. De Moor Katholieke NL q theory: checing and imposing stability of recurrent neural networs for nonlinear modelling J.A.K. Suyens, J. Vandewalle, B. De Moor Katholiee Universiteit Leuven, Dept. of Electr. Eng., ESAT-SISTA

More information

1 Smooth manifolds and Lie groups

1 Smooth manifolds and Lie groups An undergraduate approach to Lie theory Slide 1 Andrew Baker, Glasgow Glasgow, 12/11/1999 1 Smooth manifolds and Lie groups A continuous g : V 1 V 2 with V k R m k open is called smooth if it is infinitely

More information

7 Planar systems of linear ODE

7 Planar systems of linear ODE 7 Planar systems of linear ODE Here I restrict my attention to a very special class of autonomous ODE: linear ODE with constant coefficients This is arguably the only class of ODE for which explicit solution

More information

An Introduction of Basic Lie theory

An Introduction of Basic Lie theory An Introduction of Basic Lie theory Yuzheng Zhuang 02 June 2012 1 Introduction Lie theory, initially developed by Sophus Lie, is a fundamental part of mathematics. The areas it touches contain classical,

More information

An Adaptive Bayesian Network for Low-Level Image Processing

An Adaptive Bayesian Network for Low-Level Image Processing An Adaptive Bayesian Network for Low-Level Image Processing S P Luttrell Defence Research Agency, Malvern, Worcs, WR14 3PS, UK. I. INTRODUCTION Probability calculus, based on the axioms of inference, Cox

More information

L 2 Geometry of the Symplectomorphism Group

L 2 Geometry of the Symplectomorphism Group University of Notre Dame Workshop on Innite Dimensional Geometry, Vienna 2015 Outline 1 The Exponential Map on D s ω(m) 2 Existence of Multiplicity of Outline 1 The Exponential Map on D s ω(m) 2 Existence

More information

THE Hamilton equations of motion constitute a system

THE Hamilton equations of motion constitute a system Proceedings of the World Congress on Engineering 0 Vol I WCE 0, July 4-6, 0, London, U.K. Systematic Improvement of Splitting Methods for the Hamilton Equations Asif Mushtaq, Anne Kværnø, and Kåre Olaussen

More information

The characteristic equation and minimal state space realization of SISO systems in the max algebra

The characteristic equation and minimal state space realization of SISO systems in the max algebra KULeuven Department of Electrical Engineering (ESAT) SISTA Technical report 93-57a The characteristic equation and minimal state space realization of SISO systems in the max algebra B De Schutter and B

More information

A note on the characteristic equation in the max-plus algebra

A note on the characteristic equation in the max-plus algebra K.U.Leuven Department of Electrical Engineering (ESAT) SISTA Technical report 96-30 A note on the characteristic equation in the max-plus algebra B. De Schutter and B. De Moor If you want to cite this

More information

Neural Controller. Plant. Plant. Critic. evaluation signal. Reinforcement Learning Controller

Neural Controller. Plant. Plant. Critic. evaluation signal. Reinforcement Learning Controller Neural Control Theory: an Overview J.A.K. Suykens, H. Bersini Katholieke Universiteit Leuven Department of Electrical Engineering, ESAT-SISTA Kardinaal Mercierlaan 94, B-3001 Leuven (Heverlee), Belgium

More information

Departement Elektrotechniek ESAT-SISTA/TR About the choice of State Space Basis in Combined. Deterministic-Stochastic Subspace Identication 1

Departement Elektrotechniek ESAT-SISTA/TR About the choice of State Space Basis in Combined. Deterministic-Stochastic Subspace Identication 1 Katholieke Universiteit Leuven Departement Elektrotechniek ESAT-SISTA/TR 994-24 About the choice of State Space asis in Combined Deterministic-Stochastic Subspace Identication Peter Van Overschee and art

More information

Algorithmic Lie Symmetry Analysis and Group Classication for Ordinary Dierential Equations

Algorithmic Lie Symmetry Analysis and Group Classication for Ordinary Dierential Equations dmitry.lyakhov@kaust.edu.sa Symbolic Computations May 4, 2018 1 / 25 Algorithmic Lie Symmetry Analysis and Group Classication for Ordinary Dierential Equations Dmitry A. Lyakhov 1 1 Computational Sciences

More information

A First Attempt of Reservoir Pruning for Classification Problems

A First Attempt of Reservoir Pruning for Classification Problems A First Attempt of Reservoir Pruning for Classification Problems Xavier Dutoit, Hendrik Van Brussel, Marnix Nuttin Katholieke Universiteit Leuven - P.M.A. Celestijnenlaan 300b, 3000 Leuven - Belgium Abstract.

More information

Departement Elektrotechniek ESAT-SISTA/TR Minimal state space realization of SISO systems in the max algebra 1.

Departement Elektrotechniek ESAT-SISTA/TR Minimal state space realization of SISO systems in the max algebra 1. Katholieke Universiteit Leuven Departement Elektrotechniek ESAT-SISTA/TR 9- inimal state space realization of SISO systems in the max algebra 1 Bart De Schutter and Bart De oor October 199 A short version

More information

Tangent bundles, vector fields

Tangent bundles, vector fields Location Department of Mathematical Sciences,, G5-109. Main Reference: [Lee]: J.M. Lee, Introduction to Smooth Manifolds, Graduate Texts in Mathematics 218, Springer-Verlag, 2002. Homepage for the book,

More information

Akinori Sekiguchi and Yoshihiko Nakamura. Dept. of Mechano-Informatics, University of Tokyo Hongo, Bunkyo-Ku, Tokyo , Japan

Akinori Sekiguchi and Yoshihiko Nakamura. Dept. of Mechano-Informatics, University of Tokyo Hongo, Bunkyo-Ku, Tokyo , Japan The Chaotic Mobile Robot Akinori Sekiguchi and Yoshihiko Nakamura Dept. of Mechano-Informatics, University of Tokyo 7-- Hongo, Bunkyo-Ku, Tokyo -866, Japan ABSTRACT In this paper, we develop a method to

More information

Gaussian process for nonstationary time series prediction

Gaussian process for nonstationary time series prediction Computational Statistics & Data Analysis 47 (2004) 705 712 www.elsevier.com/locate/csda Gaussian process for nonstationary time series prediction Soane Brahim-Belhouari, Amine Bermak EEE Department, Hong

More information

Runge{Kutta methods on Lie groups. Hans Munthe{Kaas 1. We construct generalized Runge{Kutta methods for integration of dierential equations

Runge{Kutta methods on Lie groups. Hans Munthe{Kaas 1. We construct generalized Runge{Kutta methods for integration of dierential equations BIT xx (199x), xxx{xxx. Runge{Kutta methods on Lie groups Hans Munthe{Kaas 1 y 1 Department of Informatics, University of Bergen, Thormhlensgt. 55 N-5020, Norway. email: Hans.Munthe-Kaasii.uib.no Abstract.

More information

1 Holomorphic functions

1 Holomorphic functions Robert Oeckl CA NOTES 1 15/09/2009 1 1 Holomorphic functions 11 The complex derivative The basic objects of complex analysis are the holomorphic functions These are functions that posses a complex derivative

More information

Carnegie Mellon University Forbes Ave. Pittsburgh, PA 15213, USA. fmunos, leemon, V (x)ln + max. cost functional [3].

Carnegie Mellon University Forbes Ave. Pittsburgh, PA 15213, USA. fmunos, leemon, V (x)ln + max. cost functional [3]. Gradient Descent Approaches to Neural-Net-Based Solutions of the Hamilton-Jacobi-Bellman Equation Remi Munos, Leemon C. Baird and Andrew W. Moore Robotics Institute and Computer Science Department, Carnegie

More information

Practical symplectic partitioned Runge Kutta and Runge Kutta Nystrom methods

Practical symplectic partitioned Runge Kutta and Runge Kutta Nystrom methods Journal of Computational and Applied Mathematics 14 (00) 313 330 www.elsevier.com/locate/cam Practical symplectic partitioned Runge Kutta and Runge Kutta Nystrom methods S. Blanes a;, P.C. Moan b a Department

More information

Non-linear dynamics! in particle accelerators!

Non-linear dynamics! in particle accelerators! Non-linear dynamics! in particle accelerators! Yannis PAPAPHILIPPOU Accelerator and Beam Physics group Beams Department CERN" Cockroft Institute Lecture Courses Daresbury, UK 16-19 September 2013 1! Contents

More information

Math 1270 Honors ODE I Fall, 2008 Class notes # 14. x 0 = F (x; y) y 0 = G (x; y) u 0 = au + bv = cu + dv

Math 1270 Honors ODE I Fall, 2008 Class notes # 14. x 0 = F (x; y) y 0 = G (x; y) u 0 = au + bv = cu + dv Math 1270 Honors ODE I Fall, 2008 Class notes # 1 We have learned how to study nonlinear systems x 0 = F (x; y) y 0 = G (x; y) (1) by linearizing around equilibrium points. If (x 0 ; y 0 ) is an equilibrium

More information

ON THE NUMERICAL INTEGRATION OF ORDINARY DIFFERENTIAL EQUATIONS BY SYMMETRIC COMPOSITION METHODS. Robert I. McLachlan

ON THE NUMERICAL INTEGRATION OF ORDINARY DIFFERENTIAL EQUATIONS BY SYMMETRIC COMPOSITION METHODS. Robert I. McLachlan ON THE NUMERICAL INTEGRATION OF ORDINARY DIFFERENTIAL EQUATIONS BY SYMMETRIC COMPOSITION METHODS Robert I. McLachlan Abstract. Differential equations of the form ẋ = X = A + B are considered, where the

More information

Invariance properties in the root sensitivity of time-delay systems with double imaginary roots

Invariance properties in the root sensitivity of time-delay systems with double imaginary roots Invariance properties in the root sensitivity of time-delay systems with double imaginary roots Elias Jarlebring, Wim Michiels Department of Computer Science, KU Leuven, Celestijnenlaan A, 31 Heverlee,

More information

Linearized methods for ordinary di erential equations

Linearized methods for ordinary di erential equations Applied Mathematics and Computation 104 (1999) 109±19 www.elsevier.nl/locate/amc Linearized methods for ordinary di erential equations J.I. Ramos 1 Departamento de Lenguajes y Ciencias de la Computacion,

More information

A Remark on IVP and TVP Non-Smooth Viscosity Solutions to Hamilton-Jacobi Equations

A Remark on IVP and TVP Non-Smooth Viscosity Solutions to Hamilton-Jacobi Equations 2005 American Control Conference June 8-10, 2005. Portland, OR, USA WeB10.3 A Remark on IVP and TVP Non-Smooth Viscosity Solutions to Hamilton-Jacobi Equations Arik Melikyan, Andrei Akhmetzhanov and Naira

More information

Abstract. Jacobi curves are far going generalizations of the spaces of \Jacobi

Abstract. Jacobi curves are far going generalizations of the spaces of \Jacobi Principal Invariants of Jacobi Curves Andrei Agrachev 1 and Igor Zelenko 2 1 S.I.S.S.A., Via Beirut 2-4, 34013 Trieste, Italy and Steklov Mathematical Institute, ul. Gubkina 8, 117966 Moscow, Russia; email:

More information

The Campbell-Hausdor theorem

The Campbell-Hausdor theorem M.M. Schipper The Campbell-Hausdor theorem Bachelor's thesis, June 26, 2014 Supervisor: Dr. L.D.J. Taelman Mathematisch Instituut, Universiteit Leiden Contents 1 Introduction 3 2 Associative and Lie algebras

More information

The Erwin Schrodinger International Pasteurgasse 6/7. Institute for Mathematical Physics A-1090 Wien, Austria

The Erwin Schrodinger International Pasteurgasse 6/7. Institute for Mathematical Physics A-1090 Wien, Austria ESI The Erwin Schrodinger International Pasteurgasse 6/7 Institute for Mathematical Physics A-1090 Wien, Austria On an Extension of the 3{Dimensional Toda Lattice Mircea Puta Vienna, Preprint ESI 165 (1994)

More information

Numerical Algorithms as Dynamical Systems

Numerical Algorithms as Dynamical Systems A Study on Numerical Algorithms as Dynamical Systems Moody Chu North Carolina State University What This Study Is About? To recast many numerical algorithms as special dynamical systems, whence to derive

More information

4. Multilayer Perceptrons

4. Multilayer Perceptrons 4. Multilayer Perceptrons This is a supervised error-correction learning algorithm. 1 4.1 Introduction A multilayer feedforward network consists of an input layer, one or more hidden layers, and an output

More information

The Erwin Schrodinger International Pasteurgasse 6/7. Institute for Mathematical Physics A-1090 Wien, Austria

The Erwin Schrodinger International Pasteurgasse 6/7. Institute for Mathematical Physics A-1090 Wien, Austria ESI The Erwin Schrodinger International Pasteurgasse 6/7 Institute for Mathematical Physics A-090 Wien, Austria On the Rigid Body with Two Linear Controls Mircea Craioveanu Mircea Puta Vienna, Preprint

More information

State space transformations and state space realization in the max algebra

State space transformations and state space realization in the max algebra K.U.Leuven Department of Electrical Engineering (ESAT) SISTA Technical report 95-10 State space transformations and state space realization in the max algebra B. De Schutter and B. De Moor If you want

More information

Lie Groups for 2D and 3D Transformations

Lie Groups for 2D and 3D Transformations Lie Groups for 2D and 3D Transformations Ethan Eade Updated May 20, 2017 * 1 Introduction This document derives useful formulae for working with the Lie groups that represent transformations in 2D and

More information

Canonical Forms for BiHamiltonian Systems

Canonical Forms for BiHamiltonian Systems Canonical Forms for BiHamiltonian Systems Peter J. Olver Dedicated to the Memory of Jean-Louis Verdier BiHamiltonian systems were first defined in the fundamental paper of Magri, [5], which deduced the

More information

114 EUROPHYSICS LETTERS i) We put the question of the expansion over the set in connection with the Schrodinger operator under consideration (an accur

114 EUROPHYSICS LETTERS i) We put the question of the expansion over the set in connection with the Schrodinger operator under consideration (an accur EUROPHYSICS LETTERS 15 April 1998 Europhys. Lett., 42 (2), pp. 113-117 (1998) Schrodinger operator in an overfull set A. N. Gorban and I. V. Karlin( ) Computing Center RAS - Krasnoyars 660036, Russia (received

More information

Matrix Lie groups. and their Lie algebras. Mahmood Alaghmandan. A project in fulfillment of the requirement for the Lie algebra course

Matrix Lie groups. and their Lie algebras. Mahmood Alaghmandan. A project in fulfillment of the requirement for the Lie algebra course Matrix Lie groups and their Lie algebras Mahmood Alaghmandan A project in fulfillment of the requirement for the Lie algebra course Department of Mathematics and Statistics University of Saskatchewan March

More information

INRIA Rocquencourt, Le Chesnay Cedex (France) y Dept. of Mathematics, North Carolina State University, Raleigh NC USA

INRIA Rocquencourt, Le Chesnay Cedex (France) y Dept. of Mathematics, North Carolina State University, Raleigh NC USA Nonlinear Observer Design using Implicit System Descriptions D. von Wissel, R. Nikoukhah, S. L. Campbell y and F. Delebecque INRIA Rocquencourt, 78 Le Chesnay Cedex (France) y Dept. of Mathematics, North

More information

Symmetric Spaces Toolkit

Symmetric Spaces Toolkit Symmetric Spaces Toolkit SFB/TR12 Langeoog, Nov. 1st 7th 2007 H. Sebert, S. Mandt Contents 1 Lie Groups and Lie Algebras 2 1.1 Matrix Lie Groups........................ 2 1.2 Lie Group Homomorphisms...................

More information

Examples of the Dirac approach to dynamics of systems with constraints

Examples of the Dirac approach to dynamics of systems with constraints Physica A 290 (2001) 431 444 www.elsevier.com/locate/physa Examples of the Dirac approach to dynamics of systems with constraints Sonnet Nguyen, Lukasz A. Turski Center for Theoretical Physics, College

More information

Topics in Representation Theory: Lie Groups, Lie Algebras and the Exponential Map

Topics in Representation Theory: Lie Groups, Lie Algebras and the Exponential Map Topics in Representation Theory: Lie Groups, Lie Algebras and the Exponential Map Most of the groups we will be considering this semester will be matrix groups, i.e. subgroups of G = Aut(V ), the group

More information

DK-2800 Lyngby, Denmark, Mercierlaan 94, B{3001 Heverlee, Belgium,

DK-2800 Lyngby, Denmark, Mercierlaan 94, B{3001 Heverlee, Belgium, The Interpolation Attack on Block Ciphers? Thomas Jakobsen 1 and Lars R. Knudsen 2 1 Department of Mathematics, Building 303, Technical University of Denmark, DK-2800 Lyngby, Denmark, email:jakobsen@mat.dtu.dk.

More information

Spurious Chaotic Solutions of Dierential. Equations. Sigitas Keras. September Department of Applied Mathematics and Theoretical Physics

Spurious Chaotic Solutions of Dierential. Equations. Sigitas Keras. September Department of Applied Mathematics and Theoretical Physics UNIVERSITY OF CAMBRIDGE Numerical Analysis Reports Spurious Chaotic Solutions of Dierential Equations Sigitas Keras DAMTP 994/NA6 September 994 Department of Applied Mathematics and Theoretical Physics

More information

Lecture 2: Review of Prerequisites. Table of contents

Lecture 2: Review of Prerequisites. Table of contents Math 348 Fall 217 Lecture 2: Review of Prerequisites Disclaimer. As we have a textbook, this lecture note is for guidance and supplement only. It should not be relied on when preparing for exams. In this

More information

Since a matrix is invertible if and only if its determinant is nonzero, we can also write. GL(n) = {A R n n : deta 0}. (2)

Since a matrix is invertible if and only if its determinant is nonzero, we can also write. GL(n) = {A R n n : deta 0}. (2) MATHEMATICS 7302 (Analytical Dynamics) YEAR 2016 2017, TERM 2 HANDOUT #13: INTRODUCTION TO THE ROTATION GROUP 1 Lie groups As mentioned in Handout #11, the general framework for studying symmetry in mathematics

More information

Now an ane connection? in the frame bundle F is dened as follows. Denition 2 An ane connection? in the frame bundle F is dened in terms of the Lie alg

Now an ane connection? in the frame bundle F is dened as follows. Denition 2 An ane connection? in the frame bundle F is dened in terms of the Lie alg Modern Geometry of Image Formation Alfons H. Salden INRIA Sophia Antipolis, 2004 route des Lucioles, BP 93 F-06902 Sophia-Antipolis Cedex, France e-mail Alfons.Salden@sophia.inria.fr June 26, 1997 Abstract

More information

SYMMETRIES OF SECOND-ORDER DIFFERENTIAL EQUATIONS AND DECOUPLING

SYMMETRIES OF SECOND-ORDER DIFFERENTIAL EQUATIONS AND DECOUPLING SYMMETRIES OF SECOND-ORDER DIFFERENTIAL EQUATIONS AND DECOUPLING W. Sarlet and E. Martínez Instituut voor Theoretische Mechanica, Universiteit Gent Krijgslaan 281, B-9000 Gent, Belgium Departamento de

More information

Mathematics Research Report No. MRR 003{96, HIGH RESOLUTION POTENTIAL FLOW METHODS IN OIL EXPLORATION Stephen Roberts 1 and Stephan Matthai 2 3rd Febr

Mathematics Research Report No. MRR 003{96, HIGH RESOLUTION POTENTIAL FLOW METHODS IN OIL EXPLORATION Stephen Roberts 1 and Stephan Matthai 2 3rd Febr HIGH RESOLUTION POTENTIAL FLOW METHODS IN OIL EXPLORATION Stephen Roberts and Stephan Matthai Mathematics Research Report No. MRR 003{96, Mathematics Research Report No. MRR 003{96, HIGH RESOLUTION POTENTIAL

More information

High-order actions and their applications to honor our friend and

High-order actions and their applications to honor our friend and High-order actions and their applications to honor our friend and collaborator Siu A. Chin Universitat Politecnica de Catalunya, Barcelona, Spain Lecture: Magnus Expansion and Suzuki s Method Outline of

More information

u n 2 4 u n 36 u n 1, n 1.

u n 2 4 u n 36 u n 1, n 1. Exercise 1 Let (u n ) be the sequence defined by Set v n = u n 1 x+ u n and f (x) = 4 x. 1. Solve the equations f (x) = 1 and f (x) =. u 0 = 0, n Z +, u n+1 = u n + 4 u n.. Prove that if u n < 1, then

More information

Defense Technical Information Center Compilation Part Notice

Defense Technical Information Center Compilation Part Notice UNCLASSIFIED Defense Technical Information Center Compilation Part Notice ADP012911 TITLE: Low Temperature Scanning Tunneling Spectroscopy of Different Individual Impurities on GaAs [110] Surface and in

More information

Novel determination of dierential-equation solutions: universal approximation method

Novel determination of dierential-equation solutions: universal approximation method Journal of Computational and Applied Mathematics 146 (2002) 443 457 www.elsevier.com/locate/cam Novel determination of dierential-equation solutions: universal approximation method Thananchai Leephakpreeda

More information

J.F. Cari~nena structures and that of groupoid. So we will nd topological groupoids, Lie groupoids, symplectic groupoids, Poisson groupoids and so on.

J.F. Cari~nena structures and that of groupoid. So we will nd topological groupoids, Lie groupoids, symplectic groupoids, Poisson groupoids and so on. Lie groupoids and algebroids in Classical and Quantum Mechanics 1 Jose F. Cari~nena Departamento de Fsica Teorica. Facultad de Ciencias. Universidad de Zaragoza, E-50009, Zaragoza, Spain. Abstract The

More information

Machine Learning Lecture 5

Machine Learning Lecture 5 Machine Learning Lecture 5 Linear Discriminant Functions 26.10.2017 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Course Outline Fundamentals Bayes Decision Theory

More information

Nonlinear Waves On the Geometric Algebra

Nonlinear Waves On the Geometric Algebra Preprint Nonlinear Waves On the Geometric Algebra Carl Brannen (Dated: December, 004) Soliton solutions of nonlinear wave equations are postulated as a model for the elementary chiral fermions. The Dirac

More information

NONLINEAR CLASSIFICATION AND REGRESSION. J. Elder CSE 4404/5327 Introduction to Machine Learning and Pattern Recognition

NONLINEAR CLASSIFICATION AND REGRESSION. J. Elder CSE 4404/5327 Introduction to Machine Learning and Pattern Recognition NONLINEAR CLASSIFICATION AND REGRESSION Nonlinear Classification and Regression: Outline 2 Multi-Layer Perceptrons The Back-Propagation Learning Algorithm Generalized Linear Models Radial Basis Function

More information

output dimension input dimension Gaussian evidence Gaussian Gaussian evidence evidence from t +1 inputs and outputs at time t x t+2 x t-1 x t+1

output dimension input dimension Gaussian evidence Gaussian Gaussian evidence evidence from t +1 inputs and outputs at time t x t+2 x t-1 x t+1 To appear in M. S. Kearns, S. A. Solla, D. A. Cohn, (eds.) Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 999. Learning Nonlinear Dynamical Systems using an EM Algorithm Zoubin

More information

Ecient Higher-order Neural Networks. for Classication and Function Approximation. Joydeep Ghosh and Yoan Shin. The University of Texas at Austin

Ecient Higher-order Neural Networks. for Classication and Function Approximation. Joydeep Ghosh and Yoan Shin. The University of Texas at Austin Ecient Higher-order Neural Networks for Classication and Function Approximation Joydeep Ghosh and Yoan Shin Department of Electrical and Computer Engineering The University of Texas at Austin Austin, TX

More information

Series Expansions for Analytic Systems Linear in the Controls

Series Expansions for Analytic Systems Linear in the Controls IEEE Decision and Control Conference Sydney, Australia, Dec 2 Series Expansions for Analytic Systems Linear in the Controls Francesco Bullo Coordinated Science Laboratory and General Engineering Department

More information

On the construction of discrete gradients

On the construction of discrete gradients On the construction of discrete gradients Elizabeth L. Mansfield School of Mathematics, Statistics and Actuarial Science, University of Kent, Canterbury, CT2 7NF, U.K. G. Reinout W. Quispel Department

More information

The Hopf bifurcation with bounded noise

The Hopf bifurcation with bounded noise The Hopf bifurcation with bounded noise Ryan T. Botts 1, Ale Jan Homburg 2,3, and Todd R. Young 4 1 Department of Mathematical, Information & Computer Sciences, Point Loma Nazarene University, 3900 Lomaland

More information

Stability and bifurcation of a simple neural network with multiple time delays.

Stability and bifurcation of a simple neural network with multiple time delays. Fields Institute Communications Volume, 999 Stability and bifurcation of a simple neural network with multiple time delays. Sue Ann Campbell Department of Applied Mathematics University of Waterloo Waterloo

More information

Convergence of The Multigrid Method With A Wavelet. Abstract. This new coarse grid operator is constructed using the wavelet

Convergence of The Multigrid Method With A Wavelet. Abstract. This new coarse grid operator is constructed using the wavelet Convergence of The Multigrid Method With A Wavelet Coarse Grid Operator Bjorn Engquist Erding Luo y Abstract The convergence of the two-level multigrid method with a new coarse grid operator is studied.

More information

The Erwin Schrodinger International Pasteurgasse 6/7. Institute for Mathematical Physics A-1090 Wien, Austria

The Erwin Schrodinger International Pasteurgasse 6/7. Institute for Mathematical Physics A-1090 Wien, Austria ESI The Erwin Schrodinger International Pasteurgasse 6/7 Institute for Mathematical Physics A-1090 Wien, Austria On Twisted Tensor Products of Algebras Andreas Cap Hermann Schichl Jir Vanzura Vienna, Preprint

More information

Matrix factorization and minimal state space realization in the max-plus algebra

Matrix factorization and minimal state space realization in the max-plus algebra KULeuven Department of Electrical Engineering (ESAT) SISTA Technical report 96-69 Matrix factorization and minimal state space realization in the max-plus algebra B De Schutter and B De Moor If you want

More information

Discrete time Generalized Cellular. Johan A.K. Suykens and Joos Vandewalle. Katholieke Universiteit Leuven

Discrete time Generalized Cellular. Johan A.K. Suykens and Joos Vandewalle. Katholieke Universiteit Leuven Discrete time Generalized Cellular Neural Networks within NL q Theory Johan A.K. Suykens and Joos Vandewalle Katholieke Universiteit Leuven Department of Electrical Engineering, ESAT-SISTA Kardinaal Mercierlaan

More information

An introduction to Birkhoff normal form

An introduction to Birkhoff normal form An introduction to Birkhoff normal form Dario Bambusi Dipartimento di Matematica, Universitá di Milano via Saldini 50, 0133 Milano (Italy) 19.11.14 1 Introduction The aim of this note is to present an

More information

IMPROVED HIGH ORDER INTEGRATORS BASED ON MAGNUS EPANSION S. Blanes ;, F. Casas, and J. Ros 3 Abstract We build high order ecient numerical integration

IMPROVED HIGH ORDER INTEGRATORS BASED ON MAGNUS EPANSION S. Blanes ;, F. Casas, and J. Ros 3 Abstract We build high order ecient numerical integration UNIVERSITY OF CAMBRIDGE Numerical Analysis Reports Improved High Order Integrators Based on Magnus Expansion S. Blanes, F. Casas and J. Ros DAMTP 999/NA8 August, 999 Department of Applied Mathematics and

More information

Introduction to symplectic integrator methods for solving diverse dynamical equations

Introduction to symplectic integrator methods for solving diverse dynamical equations Introduction to symplectic integrator methods for solving diverse dynamical equations Siu A. Chin Department of Physics and Astronomy Texas A&M University College Station, TX, 77840, USA Third Lecture

More information

488 Solutions to the XOR Problem

488 Solutions to the XOR Problem 488 Solutions to the XOR Problem Frans M. Coetzee * eoetzee@eee.emu.edu Department of Electrical Engineering Carnegie Mellon University Pittsburgh, PA 15213 Virginia L. Stonick ginny@eee.emu.edu Department

More information

Note that for a, b G, we have L a R b = R b L a, by the associative law. We'll use e to denote the identity element of G.

Note that for a, b G, we have L a R b = R b L a, by the associative law. We'll use e to denote the identity element of G. NOTES ON PRINCIPAL BUNDLES, CONNECTIONS, AND HOMOGENEOUS SPACES LANCE D. DRAGER 1. Introduction These notes are meant to accompany talks in the Geometry Seminar at Texas Tech during the Spring Semester

More information

Artificial Neural Networks. MGS Lecture 2

Artificial Neural Networks. MGS Lecture 2 Artificial Neural Networks MGS 2018 - Lecture 2 OVERVIEW Biological Neural Networks Cell Topology: Input, Output, and Hidden Layers Functional description Cost functions Training ANNs Back-Propagation

More information

Parallel layer perceptron

Parallel layer perceptron Neurocomputing 55 (2003) 771 778 www.elsevier.com/locate/neucom Letters Parallel layer perceptron Walmir M. Caminhas, Douglas A.G. Vieira, João A. Vasconcelos Department of Electrical Engineering, Federal

More information

Acid con- Exposure dose. Resist. Dissolution. rate. profile. centration. C(x,z) R(x,z) z(x) Photoresist. z Wafer

Acid con- Exposure dose. Resist. Dissolution. rate. profile. centration. C(x,z) R(x,z) z(x) Photoresist. z Wafer Optimal temperature proles for post-exposure bake of photo-resist Anders Hansson and Stephen Boyd Information Systems Laboratory Stanford University Stanford, CA 9435{95 ABSTRACT In this paper it is shown

More information

Matrix-valued functions

Matrix-valued functions Matrix-valued functions Aim lecture: We solve some first order linear homogeneous differential equations using exponentials of matrices. Recall as in MATH2, the any function R M mn (C) : t A(t) can be

More information