Departement Elektrotechniek ESAT-SISTA/TR Dynamical System Prediction: a Lie algebraic approach for a novel. neural architecture 1
|
|
- Darren Lamb
- 6 years ago
- Views:
Transcription
1 Katholieke Universiteit Leuven Departement Elektrotechniek ESAT-SISTA/TR Dynamical System Prediction: a Lie algebraic approach for a novel neural architecture 1 Yves Moreau and Joos Vandewalle 2 May 1993 Accepted for publication in Annals of Mathematics and Articial Intelligence 1 This report is available by anonymous ftp from ftp.esat.kuleuven.ac.be in the directory pub/sista/moreau/reports/manna95 tr95-47.ps.z 2 ESAT - Katholieke Universiteit Leuven, Kardinaal Mercierlaan 94, 3001 Leuven (Heverlee), Belgium, tel 32/1/ , fax 32/1/ , moreau@esat.kuleuven.ac.be.
2 Abstract In this paper we study the problem of the prediction of autonomous continuous-time dynamical systems from discrete-time measurements of the state variables. From the fact that the system is inherently continuous in time but that the measurements are made at discrete time-intervals, we show that the predictor of such a system needs to be an invertible map from the state-space into itself. The problem then becomes one of how to approximate invertible maps. We show that standard approximation schemes do not guarantee the property of invertibility. We therefore propose a new approximation scheme which is based on the composition of invertible basis functions and which preserves invertibility. This approach can be cast in a Lie algebraic framework where the approximation is based on the use of the Baker-Campbell-Hausdor formula. The method can be implemented in a neural-like form and we illustrate it by an example. We also present a more general implementation which we call "MLP in dynamics space". Keywords: dynamical system prediction, Lie algebras, Baker-Campbell-Hausdor formula
3 1 Introduction We will study here the problem of the prediction of nonlinear dynamical systems from a set of sampled measurements of the state of the system. The nonlinear systems which we will consider are those nonlinear autonomous dynamical systems whose dynamics is continuous in time and therefore governed by a multidimensional ordinary dierential equation. However, although the dynamics of the system is continuous in time, the samples that we will use for prediction are the state variables measured at discrete time-intervals only. So, we are led to postulate a discretetime form for our prediction: ^x(k + 1) = F (x(k)). Using the fact that the underlying dynamics of the system is time-continuous, we will show that the function F used for prediction should be invertible. The problem then becomes that of building an approximation scheme specically designed for invertible maps. The main drawback of standard approximation schemes is that they are based on summing basis functions to build an approximation. However, this operation does not preserve invertibility. Using the operation of composition instead of addition, we are able to preserve invertibility. We will thus build approximation schemes based on composition of invertible basis functions. Those compositions can be presented in the framework of Lie algebra theory. Starting from a method from numerical analysis we will be able to derive a new approach to the prediction of nonlinear systems. This new method can easily be presented in a neural-like form. This work is organized as follows. In the rst section we will recall the basic concept of dynamical system theory and we will show that a predictor of a dynamical system should be invertible. In the second section we will present the elements of Lie algebra theory and how they can be applied, using the Baker-Campbell-Hausdor formula, to build approximation schemes based on compositions. In the third section we will present the network implementation of our model which we will illustrate by an example. We will also present a general model for approximation which we call "MLP in dynamics space". Finally, we will conclude. 1.1 Dynamical systems A dynamical system is a rule for the evolution over time of a set of variables. This set of variables is called the state of the system. Following the general dierential geometric denition of dynamical systems [2], the state will be identied here with some point on a manifold M. We will consider here only those systems for which the state evolves according to some ordinary dierential equation _x(t) = f(x(t)). The function f is the vector eld of the system and it associates to each point of the manifold M an element of the tangent space T x M. A solution to the ODE is a trajectory of the system. Such a system can thus be written as: M a manifold x(t) 2 M for all t f : M! T x M _x(t) = f(x(t)) (1) 1
4 1.2 The initial value problem and the existence, uniqueness and smoothness of solutions The initial value problem (IVP) simply consists in nding the trajectory passing through a given initial state x 0 : ( _x(t) = f(x(t)) x(0) = x 0 (2) A sucient condition for the existence of a unique solution to the IVP is that the vector eld f be everywhere locally Lipschitz [4]. The solution might however be dened only for small enough positive and negative times as the trajectory might escape from the manifold after some time. If we require the vector eld to be C r (1 r 1), the solution x(t) will be C r+1 with respect to time t and C r with respect to a change in initial condition x 0. From here on we will always assume the vector eld to be at least locally Lipschitz and we will assume further smoothness wherever necessary. 1.3 The ow of a dynamical system For each initial condition x 0, we can solve the initial value problem. We then decide to collect all those solutions in one function (x 0 ; t) which gives the solution of the ODE as a function of the initial condition: 8 > < >: _x = f(x(t)) ) x(t) = (x 0 ; t) (3) x(0) = x 0 is called the ow of the dynamical system. The ow is a remarkable object which possesses the group property. We refer the reader to [4] for a general presentation of the group theoretic approach to dynamical systems. If we assume, for simplicity, that the ow is dened for all times, the group property writes as follows: : R M! M (x; t + s) = ((x; t); s) 8 t; s 2 R (4) As we will see, this gives to the ow the structure of a group. We rst dene the time-t map as: ' t (x 0 ) (x 0 ; t) (5) This map ' t associates to any initial condition its image under the ow of the dierential equation after a given time t. So, we have on one hand a trajectory which is a way of looking at the evolution of a given initial condition for all possible times. And we have on the other hand a time-t map which is a way to look at the evolution of all possible initial conditions but only for one given time.the map ' t is a map from M into itself. Using the group property of the ow, we can deduce the properties of ': ' t ' s (x 0 ) = ((x 0 ; s); t) = (x 0 ; t + s) = ' t+s (x 0 ) ' 0 = I ( ' 0 (x 0 ) = (x 0 ; 0) = x 0 = I(x 0 ) ' t '?t (x0) = (x 0 ; 0) = x 0 = I(x 0 ) ) '?t = (' t )?1 composition identity inverse () 2
5 Moreover, if f is C r then ' is C r. Hence, ' is a smooth invertible map with a smooth inverse, that is, ' is a dieomorphism. Furthermore, the properties of ' show that f' t g is a group. We therefore say that a ow (x; t) is a one-parameter group of dieomorphisms ' t. 1.4 Dynamical system prediction Even if we assume that the underlying dynamics of the system is governed by an ODE, the system is generally observed only at discrete time-intervals. So, if we choose a sampling period t, we can assume a functional relationship between the current state and the next sample. The system becomes of the form: x((k + 1):t) = F (x(k:t)); x(k) 2 M (7) And, from what we have shown before, we can see that: Hence, F is a dieomorphism. F (:) = ' t (:) (8) The following gure (Fig. 1) summarizes those observations. On the left we consider a square grid of initial conditions. We then let each point of that grid follow its trajectory for a timeinterval t. After this time, all the points of the grid have moved and the shape of the grid has changed. The dieomorphism ' t is that transformation from the manifold (here, R 2 ) into itself which maps the square grid on the left into the deformed grid on the right. One can immediately see that this transformation is smooth and invertible, hence a dieomorphism. ' t (x 0 ) = (x 0 ; t) Figure 1: Dieomorphic transformation of a square grid after one time-step. So, the question of predicting a continuous-time system observed at discrete time-intervals becomes one of how we can approximate those dieomorphisms which arise from the sampling of an ODE. The basic idea is that the set of all C r dieomorphisms is also a group. In a group, the natural operation is the composition of the elements of the group. While most standard approximation schemes are based on the summation of some basis functions, this operation does not preserve the property of being a dieomorphism. On the other hand, composition does preserve this property, so we propose to build an approximation scheme based on composing basis 3
6 functions instead of adding them. Schematically, one could caricature this by saying that the transformation in Fig. 1 can be approximated by a sequence of two successive transformations. The rst transformation could for example pinch the left and right sides of the square and then be followed by a second one which would be a clockwise rotation of about 15 degrees. 2 Lie algebra theory Lie algebra theory has a long history in physics, mostly in the areas of classical mechanics [1] and partial dierential equations [8]. It is also an essential part of nonlinear system theory [5]. Our approach to system prediction can be best cast in this framework. A Lie algebra is dened as follows: Denition.1 let V be a vector space; V is a Lie algebra; with Lie bracket [:; :] : V V! V i for all A; B; A 1 ; A 2 ; C 2 V : [A; B] =?[B; A] antisymmetry [a:a; B] = a:[a; B] [A 1 + A 2 ; B] = [A 1 ; B] + [A 2 ; B] bilinearity [[A; B]; C] + [[B; C]; A] + [[C; A]; B] = 0 Jacobi identity (9) In the case where the Lie algebra is the vector space of all C r vector elds, the time-t map ' t plays a very important role. If the vector eld is A, we dene the exponential map as follows: exp(t:a) ' t. This notation is an extension of the case where the vector eld is linear in space and where the solution is given by exponentiation of the matrix. The exponential is thus a mapping from the manifold into itself. And this map depends on the underlying vector eld A and changes over time. From here on, the product of exponentials will denote the composition of the maps. One can then dene the Lie bracket by: [A; B] exp(?s:b): exp(?t:a): exp(s:b): exp(t:a) t=s=0 The bracket [A; B] is called the commutator of the vector elds as it measures the degree of noncommutativity of the ows of the vector elds ( [A; B] = 0, exp(a): exp(b) = exp(b): exp(a)). In the case where the manifold is R n, the bracket can be further particularized to: nx [A; B] i = B i j? A i j i i=1 4
7 2.1 The Baker-Campbell-Hausdor formula The Baker-Campbell-Hausdor (BCH) formula gives an expansion for the product of two exponentials of elements of the Lie algebra, see []. exp(a): exp(b) = exp(a + B [A; B] ([A; [A; B]]? [B; [B; A]]) + [A; [B; [B; A]]] + : : :) (12) The problem of predicting a dynamical system with vector eld X then becomes that of building an approximation for exp(t:x) as we have that: x(k + 1) = exp(t:x)[x(k)] (13) This problem has recently been the focus of much attention in the eld of numerical analysis for the integration of Hamiltonian dierential equations [] [7]. Suppose that the vector eld X is of the form X = A + B where we can integrate A and B directly. Then one can use the BCH formula to produce a rst-order approximation to the exponential map: BCH : exp(t:x) = exp(t:a): exp(t:b) + o(t 2 ) (14) This is the essence of the method as it shows that one can approximate an exponential map (that is the mapping arising from the solution of an ODE) by composing simpler maps. By further manipulating the BCH formula, we can show that the following leapfrog scheme is second-order. Leapfrog : exp(t:x) = exp( t :A): exp(t:b): exp(t 2 2 :A) + o(t3 ) (15) Using this leapfrog scheme as a basis element for further leapfrog schemes. Yoshida [9] showed that it was in fact possible to produce an approximation to exp(t:x) up to any order. Order p : 9k; 9w 1 ; v 1 ; : : :; w k ; v k : exp(t:x) = exp(w 1 :t:a): exp(v 1 :t:b) : : : exp(w k :t:a): exp(v k :t:b) + o(t p+1 ) (1) Forest and Ruth [3] showed that approximations could be built for more than two vector elds. Combining the two we can state that it is possible to build an approximation to the solution of a linear combination of vector elds as a product of exponential maps. X = P m i=1 a i :A i ) 9w ij : exp(t:x) = Q k j=1 Q mi=1 exp(w ij :t:a i ) + o(t p+1 ) (17) 3 Network implementation Such an approximation scheme can be easily implemented as a network as can be seen in the following gure (Fig. 2). The problem of predicting the system can now simply be solved by minimizing the prediction error of the model by tuning the weights w ij. Moreover, gradients can easily be computed which allows for a gradient-descent, backpropagation-like implementation or any other gradient based optimization method. 5
8 ^x(k + 1) exp(w 22 A 2 ) exp(w 21 A 1 ) exp(w 12 A 2 ) exp(w 11 A 1 ) x(k) w 22 w 21 w 12 w 11 Figure 2: Implementation of the composition network. 3.1 Example We will now present an example to illustrate how this method can be applied. We will do this by looking at the Vanderpol oscillator. This system is described by the following second-order nonlinear dierential equation: x? :(1? x 2 ): _x + x = 0 (18) It can be directly written as a rst-order system in state-space form: ( _x = y _y =?x + :(1? x 2 ):y This system can be seen as a linear combination of two vector elds which can be solved analytically:!!! _x y 0 = +: _y?x (1? x 2 ):y (20) = A 1 + :A 2 One can then build a predictor according to the architecture just presented (Fig. 2), simply choosing an appropriate number of layers. One then nds the appropriate weights w ij by minimizing the error on the training set. One interesting property of this method is that we were able to use the a priori knowledge we had about the system. Such a feature is uncommon in the eld of system prediction using neural networks. This method allowed us to build a predictor for the Vanderpol oscillator in the case where the parameter was unknown. Using the same method we were able to build a predictor for the Lorentz attractor in the case where the parameters were also unknown. (19)
9 3.2 MLP in dynamics space If we decide to approximate the vector eld of our system by a multi-layer perceptron, one can derive a composition network to implement this. The form of the vector eld is: X(x) = nx j=1 ~c j :( ~ b j :x + b j0 ) = X j=1 where (x) = tanh(x) and ~c j ; ~ b j 2 R d+1 ; j = 1; :::; n. A ~c j; ~ b j j (x) (21) But the dierential equation _x = (x) can be solved explicitly in the one-dimensional case. We can use this to explicitly integrate the multidimensional system _x = A ~c;~ b (x) for any value of the parameters ~c j ; ~ b j. So, we can design a network of the following form: ^x(k + 1) = F (x(k)) = exp(w k :A ~c k; ~ b k ) : : : exp(w 1 :A ~c 1; ~ b 1 )(x(k)) (22) We call this an \MLP in dynamics space" as the MLP is implicitly used to parametrize the vector eld. Unfortunately, preliminary simulations have shown so far that it is very hard to nd the optimal coecients of such a model using only brute force gradient optimization. Further insight is needed in order to nd less computationally demanding methods for the training of such a network. 4 Conclusions We have studied the problem of predicting an autonomous continuous-time nonlinear dynamical system from discrete-time measurements of its state. We showed that because the predictor must have the form of an invertible map, it is interesting to have an approximation scheme which has this property. This can be achieved by using compositions instead of additions in the approximation. The theory of Lie algebras provided a framework to develop such a method, in a fashion similar to what is done for some numerical methods of integration of ordinary dierential equations. We showed that this resulted in a neural-like architecture which we illustrated by a simple example, and that it could be further extended to some form of "MLP in dynamics space". Further work will be primarily devoted to practical applications of this scheme, especially to the design of ecient training algorithms. Acknowledgments This research work was carried out at the ESAT laboratory of the Katholieke Universiteit Leuven, in the framework of a Concerted Action Project of the Flemish Community, entitled Model-based Information Processing Systems GOA-MIPS, and within the framework of the Belgian program on inter-university attraction poles (IUAP-17) initiated by the Belgian State, Prime Minister's Oce for Science, Technology and Culture, and within the framework of the European Commission Human Capital and Mobility SIMONET (System Identication and Modeling Network). The scientic responsibility rests with its authors. Yves Moreau is a research assistant of the N.F.W.O. (Belgian National Fund for Scientic Research). 7
10 References [1] V.I. Arnold, Mathematical methods of classical mechanics., Springer-Verlag, New York, [2] D.R.J.Chillingworth, Dierential topology with a view to applications., Pitman Publishing, London, [3] E. Forest, R. Ruth, Fourth-order symplectic integration, Physica D, 43 (1990) [4] M.C. Irwin, Smooth Dynamical Systems., Academic Press, New York, [5] A. Isidori, Nonlinear control theory, Springer-Verlag, New York, [] P.-V. Kosele, Calcul formel pour les methodes de Lie en mecanique hamiltonienne, Ph.D. thesis, Ecole Polytechnique, Paris, [7] R.I. McLachlan, On the numerical integration of ordinary dierential equations by symmetric composition methods, SIAM J. Sci. Comput., 1(1) (1995) [8] P.J Olver, Applications of Lie groups to dierential equations., Springer-Verlag, New York, [9] H. Yoshida, Construction of higher order symplectic integrators, Phys. Lett. A, 150 (1990)
Accelerate Implementation of Forwaring Control Laws using Composition Methos Yves Moreau an Roolphe Sepulchre June 1997 Abstract We use a metho of int
Katholieke Universiteit Leuven Departement Elektrotechniek ESAT-SISTA/TR 1997-11 Accelerate Implementation of Forwaring Control Laws using Composition Methos 1 Yves Moreau, Roolphe Sepulchre, Joos Vanewalle
More informationCOMPOSITION METHODS FOR THE SIMULATION OF ARRAYS OF CHUA S CIRCUITS
International Journal of Bifurcation and Chaos, Vol. 9, No. 4 (1999) 723 733 c World Scientific Publishing Company COMPOSITION METHODS FOR THE SIMULATION OF ARRAYS OF CHUA S CIRCUITS YVES MOREAU and JOOS
More informationOptimal trac light control for a single intersection Bart De Schutter and Bart De Moor y ESAT-SISTA, KU Leuven, Kardinaal Mercierlaan 94, B-3 Leuven (
Katholieke Universiteit Leuven Departement Elektrotechniek ESAT-SISTA/TR 97- Optimal trac light control for a single intersection Bart De Schutter and Bart De Moor Proceedings of the 997 International
More informationx 104
Departement Elektrotechniek ESAT-SISTA/TR 98-3 Identication of the circulant modulated Poisson process: a time domain approach Katrien De Cock, Tony Van Gestel and Bart De Moor 2 April 998 Submitted for
More informationThe Linear Dynamic Complementarity Problem is a special case of the Extended Linear Complementarity Problem B. De Schutter 1 B. De Moor ESAT-SISTA, K.
Katholieke Universiteit Leuven Departement Elektrotechniek ESAT-SISTA/TR 9-1 The Linear Dynamic Complementarity Problem is a special case of the Extended Linear Complementarity Problem 1 Bart De Schutter
More informationOn max-algebraic models for transportation networks
K.U.Leuven Department of Electrical Engineering (ESAT) SISTA Technical report 98-00 On max-algebraic models for transportation networks R. de Vries, B. De Schutter, and B. De Moor If you want to cite this
More informationEmbedding Recurrent Neural Networks into Predator-Prey Models Yves Moreau, St phane Louies 2, Joos Vandewalle, L on renig 2 Katholieke Universiteit Le
Katholieke Universiteit Leuven Departement Elektrotechniek ESAT-SISTA/TR 998-2 Embedding Recurrent Neural Networks into Predator-Prey Models Yves Moreau 2, Stephane Louies 3, Joos Vandewalle 2, Leon renig
More informationDepartement Elektrotechniek ESAT-SISTA/TR 98- Stochastic System Identication for ATM Network Trac Models: a Time Domain Approach Katrien De Cock and Bart De Moor April 998 Accepted for publication in roceedings
More informationElementary Approximation of Exponentials of. Frederic Jean & Pierre{Vincent Koseleff. Equipe Analyse Algebrique, Institut de Mathematiques
Elementary Approximation of Exponentials of Lie Polynomials? Frederic Jean & Pierre{Vincent Koseleff Equipe Analyse Algebrique, Institut de Mathematiques Universite Paris 6, Case 8 4 place Jussieu, F-755
More informationAbstract. In this paper we study the performance of a symplectic numerical integrator
Geometric Numerical Integration Applied to The Elastic Pendulum at Higher Order Resonance J. M. Tuwankotta y G.R.W. Quispel E-mail: tuwankotta@math.uu.nl Email: R.Quispel@latrobe.edu.au Mathematisch Instituut,
More informationMaster-Slave Synchronization using. Dynamic Output Feedback. Kardinaal Mercierlaan 94, B-3001 Leuven (Heverlee), Belgium
Master-Slave Synchronization using Dynamic Output Feedback J.A.K. Suykens 1, P.F. Curran and L.O. Chua 1 Katholieke Universiteit Leuven, Dept. of Electr. Eng., ESAT-SISTA Kardinaal Mercierlaan 94, B-1
More information28344 Bremen, Germany. hdullin or
Ecient Calculation of Actions H.R. Dullin A. Wittek Institut fur Theoretische Physik Universitat Bremen Postfach 330440 28344 Bremen, Germany Phone: +49(0)421-218-2341 Fax: +49(0)421-218-4869 Email: hdullin
More informationTHEODORE VORONOV DIFFERENTIABLE MANIFOLDS. Fall Last updated: November 26, (Under construction.)
4 Vector fields Last updated: November 26, 2009. (Under construction.) 4.1 Tangent vectors as derivations After we have introduced topological notions, we can come back to analysis on manifolds. Let M
More informationZIG-ZAG DYNAMICAL SYSTEMS AND THE BAKER-CAMPBELL-HAUSDORFF FORMULA. Alois Klc Pavel Pokorny Jan Rehacek. (Communicated by Milan Medved')
Math. Slovaca 5 () No. 1 nn{nn aoc Mathematical Institute Slovak Academy of Sciences ZIG-ZAG DYNAMICAL SYSTEMS AND THE BAKER-CAMPBELL-HAUSDORFF FORMULA Alois Klc Pavel Pokorny Jan Rehacek (Communicated
More informationThe Erwin Schrodinger International Boltzmanngasse 9. Institute for Mathematical Physics A-1090 Wien, Austria
ESI The Erwin Schrodinger International Boltzmanngasse 9 Institute for Mathematical Physics A-1090 Wien, Austria Noncommutative Contact Algebras Hideki Omori Yoshiaki Maeda Naoya Miyazaki Akira Yoshioka
More informationOptimal traffic light control for a single intersection
KULeuven Department of Electrical Engineering (ESAT) SISTA Technical report 97- Optimal traffic light control for a single intersection B De Schutter and B De Moor If you want to cite this report, please
More informationREMARKS ON THE TIME-OPTIMAL CONTROL OF A CLASS OF HAMILTONIAN SYSTEMS. Eduardo D. Sontag. SYCON - Rutgers Center for Systems and Control
REMARKS ON THE TIME-OPTIMAL CONTROL OF A CLASS OF HAMILTONIAN SYSTEMS Eduardo D. Sontag SYCON - Rutgers Center for Systems and Control Department of Mathematics, Rutgers University, New Brunswick, NJ 08903
More informationSVD-based optimal ltering with applications to noise reduction in speech signals Simon Doclo ESAT - SISTA, Katholieke Universiteit Leuven Kardinaal Me
Departement Elektrotechniek ESAT-SISTA/TR 999- SVD-based Optimal Filtering with Applications to Noise Reduction in Speech Signals Simon Doclo, Marc Moonen April, 999 Internal report This report is available
More informationGeneralized Shifted Inverse Iterations on Grassmann Manifolds 1
Proceedings of the Sixteenth International Symposium on Mathematical Networks and Systems (MTNS 2004), Leuven, Belgium Generalized Shifted Inverse Iterations on Grassmann Manifolds 1 J. Jordan α, P.-A.
More informationTHE RICCI BRACKET FLOW FOR SOLVABLE LIE GROUPS
THE RICCI BRACKET FLOW FOR SOLVABLE LIE GROUPS KEVIN KOZLOWSKI AND BRENNAN VINCENT Abstract. The Ricci bracket flow is a geometric evolution on Lie algebras which is related to the Ricci flow on the corresponding
More informationOn Mean Curvature Diusion in Nonlinear Image Filtering. Adel I. El-Fallah and Gary E. Ford. University of California, Davis. Davis, CA
On Mean Curvature Diusion in Nonlinear Image Filtering Adel I. El-Fallah and Gary E. Ford CIPIC, Center for Image Processing and Integrated Computing University of California, Davis Davis, CA 95616 Abstract
More informationbelow, kernel PCA Eigenvectors, and linear combinations thereof. For the cases where the pre-image does exist, we can provide a means of constructing
Kernel PCA Pattern Reconstruction via Approximate Pre-Images Bernhard Scholkopf, Sebastian Mika, Alex Smola, Gunnar Ratsch, & Klaus-Robert Muller GMD FIRST, Rudower Chaussee 5, 12489 Berlin, Germany fbs,
More informationDefinition 5.1. A vector field v on a manifold M is map M T M such that for all x M, v(x) T x M.
5 Vector fields Last updated: March 12, 2012. 5.1 Definition and general properties We first need to define what a vector field is. Definition 5.1. A vector field v on a manifold M is map M T M such that
More informationNL q theory: checking and imposing stability of recurrent neural networks for nonlinear modelling J.A.K. Suykens, J. Vandewalle, B. De Moor Katholieke
NL q theory: checing and imposing stability of recurrent neural networs for nonlinear modelling J.A.K. Suyens, J. Vandewalle, B. De Moor Katholiee Universiteit Leuven, Dept. of Electr. Eng., ESAT-SISTA
More information1 Smooth manifolds and Lie groups
An undergraduate approach to Lie theory Slide 1 Andrew Baker, Glasgow Glasgow, 12/11/1999 1 Smooth manifolds and Lie groups A continuous g : V 1 V 2 with V k R m k open is called smooth if it is infinitely
More information7 Planar systems of linear ODE
7 Planar systems of linear ODE Here I restrict my attention to a very special class of autonomous ODE: linear ODE with constant coefficients This is arguably the only class of ODE for which explicit solution
More informationAn Introduction of Basic Lie theory
An Introduction of Basic Lie theory Yuzheng Zhuang 02 June 2012 1 Introduction Lie theory, initially developed by Sophus Lie, is a fundamental part of mathematics. The areas it touches contain classical,
More informationAn Adaptive Bayesian Network for Low-Level Image Processing
An Adaptive Bayesian Network for Low-Level Image Processing S P Luttrell Defence Research Agency, Malvern, Worcs, WR14 3PS, UK. I. INTRODUCTION Probability calculus, based on the axioms of inference, Cox
More informationL 2 Geometry of the Symplectomorphism Group
University of Notre Dame Workshop on Innite Dimensional Geometry, Vienna 2015 Outline 1 The Exponential Map on D s ω(m) 2 Existence of Multiplicity of Outline 1 The Exponential Map on D s ω(m) 2 Existence
More informationTHE Hamilton equations of motion constitute a system
Proceedings of the World Congress on Engineering 0 Vol I WCE 0, July 4-6, 0, London, U.K. Systematic Improvement of Splitting Methods for the Hamilton Equations Asif Mushtaq, Anne Kværnø, and Kåre Olaussen
More informationThe characteristic equation and minimal state space realization of SISO systems in the max algebra
KULeuven Department of Electrical Engineering (ESAT) SISTA Technical report 93-57a The characteristic equation and minimal state space realization of SISO systems in the max algebra B De Schutter and B
More informationA note on the characteristic equation in the max-plus algebra
K.U.Leuven Department of Electrical Engineering (ESAT) SISTA Technical report 96-30 A note on the characteristic equation in the max-plus algebra B. De Schutter and B. De Moor If you want to cite this
More informationNeural Controller. Plant. Plant. Critic. evaluation signal. Reinforcement Learning Controller
Neural Control Theory: an Overview J.A.K. Suykens, H. Bersini Katholieke Universiteit Leuven Department of Electrical Engineering, ESAT-SISTA Kardinaal Mercierlaan 94, B-3001 Leuven (Heverlee), Belgium
More informationDepartement Elektrotechniek ESAT-SISTA/TR About the choice of State Space Basis in Combined. Deterministic-Stochastic Subspace Identication 1
Katholieke Universiteit Leuven Departement Elektrotechniek ESAT-SISTA/TR 994-24 About the choice of State Space asis in Combined Deterministic-Stochastic Subspace Identication Peter Van Overschee and art
More informationAlgorithmic Lie Symmetry Analysis and Group Classication for Ordinary Dierential Equations
dmitry.lyakhov@kaust.edu.sa Symbolic Computations May 4, 2018 1 / 25 Algorithmic Lie Symmetry Analysis and Group Classication for Ordinary Dierential Equations Dmitry A. Lyakhov 1 1 Computational Sciences
More informationA First Attempt of Reservoir Pruning for Classification Problems
A First Attempt of Reservoir Pruning for Classification Problems Xavier Dutoit, Hendrik Van Brussel, Marnix Nuttin Katholieke Universiteit Leuven - P.M.A. Celestijnenlaan 300b, 3000 Leuven - Belgium Abstract.
More informationDepartement Elektrotechniek ESAT-SISTA/TR Minimal state space realization of SISO systems in the max algebra 1.
Katholieke Universiteit Leuven Departement Elektrotechniek ESAT-SISTA/TR 9- inimal state space realization of SISO systems in the max algebra 1 Bart De Schutter and Bart De oor October 199 A short version
More informationTangent bundles, vector fields
Location Department of Mathematical Sciences,, G5-109. Main Reference: [Lee]: J.M. Lee, Introduction to Smooth Manifolds, Graduate Texts in Mathematics 218, Springer-Verlag, 2002. Homepage for the book,
More informationAkinori Sekiguchi and Yoshihiko Nakamura. Dept. of Mechano-Informatics, University of Tokyo Hongo, Bunkyo-Ku, Tokyo , Japan
The Chaotic Mobile Robot Akinori Sekiguchi and Yoshihiko Nakamura Dept. of Mechano-Informatics, University of Tokyo 7-- Hongo, Bunkyo-Ku, Tokyo -866, Japan ABSTRACT In this paper, we develop a method to
More informationGaussian process for nonstationary time series prediction
Computational Statistics & Data Analysis 47 (2004) 705 712 www.elsevier.com/locate/csda Gaussian process for nonstationary time series prediction Soane Brahim-Belhouari, Amine Bermak EEE Department, Hong
More informationRunge{Kutta methods on Lie groups. Hans Munthe{Kaas 1. We construct generalized Runge{Kutta methods for integration of dierential equations
BIT xx (199x), xxx{xxx. Runge{Kutta methods on Lie groups Hans Munthe{Kaas 1 y 1 Department of Informatics, University of Bergen, Thormhlensgt. 55 N-5020, Norway. email: Hans.Munthe-Kaasii.uib.no Abstract.
More information1 Holomorphic functions
Robert Oeckl CA NOTES 1 15/09/2009 1 1 Holomorphic functions 11 The complex derivative The basic objects of complex analysis are the holomorphic functions These are functions that posses a complex derivative
More informationCarnegie Mellon University Forbes Ave. Pittsburgh, PA 15213, USA. fmunos, leemon, V (x)ln + max. cost functional [3].
Gradient Descent Approaches to Neural-Net-Based Solutions of the Hamilton-Jacobi-Bellman Equation Remi Munos, Leemon C. Baird and Andrew W. Moore Robotics Institute and Computer Science Department, Carnegie
More informationPractical symplectic partitioned Runge Kutta and Runge Kutta Nystrom methods
Journal of Computational and Applied Mathematics 14 (00) 313 330 www.elsevier.com/locate/cam Practical symplectic partitioned Runge Kutta and Runge Kutta Nystrom methods S. Blanes a;, P.C. Moan b a Department
More informationNon-linear dynamics! in particle accelerators!
Non-linear dynamics! in particle accelerators! Yannis PAPAPHILIPPOU Accelerator and Beam Physics group Beams Department CERN" Cockroft Institute Lecture Courses Daresbury, UK 16-19 September 2013 1! Contents
More informationMath 1270 Honors ODE I Fall, 2008 Class notes # 14. x 0 = F (x; y) y 0 = G (x; y) u 0 = au + bv = cu + dv
Math 1270 Honors ODE I Fall, 2008 Class notes # 1 We have learned how to study nonlinear systems x 0 = F (x; y) y 0 = G (x; y) (1) by linearizing around equilibrium points. If (x 0 ; y 0 ) is an equilibrium
More informationON THE NUMERICAL INTEGRATION OF ORDINARY DIFFERENTIAL EQUATIONS BY SYMMETRIC COMPOSITION METHODS. Robert I. McLachlan
ON THE NUMERICAL INTEGRATION OF ORDINARY DIFFERENTIAL EQUATIONS BY SYMMETRIC COMPOSITION METHODS Robert I. McLachlan Abstract. Differential equations of the form ẋ = X = A + B are considered, where the
More informationInvariance properties in the root sensitivity of time-delay systems with double imaginary roots
Invariance properties in the root sensitivity of time-delay systems with double imaginary roots Elias Jarlebring, Wim Michiels Department of Computer Science, KU Leuven, Celestijnenlaan A, 31 Heverlee,
More informationLinearized methods for ordinary di erential equations
Applied Mathematics and Computation 104 (1999) 109±19 www.elsevier.nl/locate/amc Linearized methods for ordinary di erential equations J.I. Ramos 1 Departamento de Lenguajes y Ciencias de la Computacion,
More informationA Remark on IVP and TVP Non-Smooth Viscosity Solutions to Hamilton-Jacobi Equations
2005 American Control Conference June 8-10, 2005. Portland, OR, USA WeB10.3 A Remark on IVP and TVP Non-Smooth Viscosity Solutions to Hamilton-Jacobi Equations Arik Melikyan, Andrei Akhmetzhanov and Naira
More informationAbstract. Jacobi curves are far going generalizations of the spaces of \Jacobi
Principal Invariants of Jacobi Curves Andrei Agrachev 1 and Igor Zelenko 2 1 S.I.S.S.A., Via Beirut 2-4, 34013 Trieste, Italy and Steklov Mathematical Institute, ul. Gubkina 8, 117966 Moscow, Russia; email:
More informationThe Campbell-Hausdor theorem
M.M. Schipper The Campbell-Hausdor theorem Bachelor's thesis, June 26, 2014 Supervisor: Dr. L.D.J. Taelman Mathematisch Instituut, Universiteit Leiden Contents 1 Introduction 3 2 Associative and Lie algebras
More informationThe Erwin Schrodinger International Pasteurgasse 6/7. Institute for Mathematical Physics A-1090 Wien, Austria
ESI The Erwin Schrodinger International Pasteurgasse 6/7 Institute for Mathematical Physics A-1090 Wien, Austria On an Extension of the 3{Dimensional Toda Lattice Mircea Puta Vienna, Preprint ESI 165 (1994)
More informationNumerical Algorithms as Dynamical Systems
A Study on Numerical Algorithms as Dynamical Systems Moody Chu North Carolina State University What This Study Is About? To recast many numerical algorithms as special dynamical systems, whence to derive
More information4. Multilayer Perceptrons
4. Multilayer Perceptrons This is a supervised error-correction learning algorithm. 1 4.1 Introduction A multilayer feedforward network consists of an input layer, one or more hidden layers, and an output
More informationThe Erwin Schrodinger International Pasteurgasse 6/7. Institute for Mathematical Physics A-1090 Wien, Austria
ESI The Erwin Schrodinger International Pasteurgasse 6/7 Institute for Mathematical Physics A-090 Wien, Austria On the Rigid Body with Two Linear Controls Mircea Craioveanu Mircea Puta Vienna, Preprint
More informationState space transformations and state space realization in the max algebra
K.U.Leuven Department of Electrical Engineering (ESAT) SISTA Technical report 95-10 State space transformations and state space realization in the max algebra B. De Schutter and B. De Moor If you want
More informationLie Groups for 2D and 3D Transformations
Lie Groups for 2D and 3D Transformations Ethan Eade Updated May 20, 2017 * 1 Introduction This document derives useful formulae for working with the Lie groups that represent transformations in 2D and
More informationCanonical Forms for BiHamiltonian Systems
Canonical Forms for BiHamiltonian Systems Peter J. Olver Dedicated to the Memory of Jean-Louis Verdier BiHamiltonian systems were first defined in the fundamental paper of Magri, [5], which deduced the
More information114 EUROPHYSICS LETTERS i) We put the question of the expansion over the set in connection with the Schrodinger operator under consideration (an accur
EUROPHYSICS LETTERS 15 April 1998 Europhys. Lett., 42 (2), pp. 113-117 (1998) Schrodinger operator in an overfull set A. N. Gorban and I. V. Karlin( ) Computing Center RAS - Krasnoyars 660036, Russia (received
More informationMatrix Lie groups. and their Lie algebras. Mahmood Alaghmandan. A project in fulfillment of the requirement for the Lie algebra course
Matrix Lie groups and their Lie algebras Mahmood Alaghmandan A project in fulfillment of the requirement for the Lie algebra course Department of Mathematics and Statistics University of Saskatchewan March
More informationINRIA Rocquencourt, Le Chesnay Cedex (France) y Dept. of Mathematics, North Carolina State University, Raleigh NC USA
Nonlinear Observer Design using Implicit System Descriptions D. von Wissel, R. Nikoukhah, S. L. Campbell y and F. Delebecque INRIA Rocquencourt, 78 Le Chesnay Cedex (France) y Dept. of Mathematics, North
More informationSymmetric Spaces Toolkit
Symmetric Spaces Toolkit SFB/TR12 Langeoog, Nov. 1st 7th 2007 H. Sebert, S. Mandt Contents 1 Lie Groups and Lie Algebras 2 1.1 Matrix Lie Groups........................ 2 1.2 Lie Group Homomorphisms...................
More informationExamples of the Dirac approach to dynamics of systems with constraints
Physica A 290 (2001) 431 444 www.elsevier.com/locate/physa Examples of the Dirac approach to dynamics of systems with constraints Sonnet Nguyen, Lukasz A. Turski Center for Theoretical Physics, College
More informationTopics in Representation Theory: Lie Groups, Lie Algebras and the Exponential Map
Topics in Representation Theory: Lie Groups, Lie Algebras and the Exponential Map Most of the groups we will be considering this semester will be matrix groups, i.e. subgroups of G = Aut(V ), the group
More informationDK-2800 Lyngby, Denmark, Mercierlaan 94, B{3001 Heverlee, Belgium,
The Interpolation Attack on Block Ciphers? Thomas Jakobsen 1 and Lars R. Knudsen 2 1 Department of Mathematics, Building 303, Technical University of Denmark, DK-2800 Lyngby, Denmark, email:jakobsen@mat.dtu.dk.
More informationSpurious Chaotic Solutions of Dierential. Equations. Sigitas Keras. September Department of Applied Mathematics and Theoretical Physics
UNIVERSITY OF CAMBRIDGE Numerical Analysis Reports Spurious Chaotic Solutions of Dierential Equations Sigitas Keras DAMTP 994/NA6 September 994 Department of Applied Mathematics and Theoretical Physics
More informationLecture 2: Review of Prerequisites. Table of contents
Math 348 Fall 217 Lecture 2: Review of Prerequisites Disclaimer. As we have a textbook, this lecture note is for guidance and supplement only. It should not be relied on when preparing for exams. In this
More informationSince a matrix is invertible if and only if its determinant is nonzero, we can also write. GL(n) = {A R n n : deta 0}. (2)
MATHEMATICS 7302 (Analytical Dynamics) YEAR 2016 2017, TERM 2 HANDOUT #13: INTRODUCTION TO THE ROTATION GROUP 1 Lie groups As mentioned in Handout #11, the general framework for studying symmetry in mathematics
More informationNow an ane connection? in the frame bundle F is dened as follows. Denition 2 An ane connection? in the frame bundle F is dened in terms of the Lie alg
Modern Geometry of Image Formation Alfons H. Salden INRIA Sophia Antipolis, 2004 route des Lucioles, BP 93 F-06902 Sophia-Antipolis Cedex, France e-mail Alfons.Salden@sophia.inria.fr June 26, 1997 Abstract
More informationSYMMETRIES OF SECOND-ORDER DIFFERENTIAL EQUATIONS AND DECOUPLING
SYMMETRIES OF SECOND-ORDER DIFFERENTIAL EQUATIONS AND DECOUPLING W. Sarlet and E. Martínez Instituut voor Theoretische Mechanica, Universiteit Gent Krijgslaan 281, B-9000 Gent, Belgium Departamento de
More informationMathematics Research Report No. MRR 003{96, HIGH RESOLUTION POTENTIAL FLOW METHODS IN OIL EXPLORATION Stephen Roberts 1 and Stephan Matthai 2 3rd Febr
HIGH RESOLUTION POTENTIAL FLOW METHODS IN OIL EXPLORATION Stephen Roberts and Stephan Matthai Mathematics Research Report No. MRR 003{96, Mathematics Research Report No. MRR 003{96, HIGH RESOLUTION POTENTIAL
More informationHigh-order actions and their applications to honor our friend and
High-order actions and their applications to honor our friend and collaborator Siu A. Chin Universitat Politecnica de Catalunya, Barcelona, Spain Lecture: Magnus Expansion and Suzuki s Method Outline of
More informationu n 2 4 u n 36 u n 1, n 1.
Exercise 1 Let (u n ) be the sequence defined by Set v n = u n 1 x+ u n and f (x) = 4 x. 1. Solve the equations f (x) = 1 and f (x) =. u 0 = 0, n Z +, u n+1 = u n + 4 u n.. Prove that if u n < 1, then
More informationDefense Technical Information Center Compilation Part Notice
UNCLASSIFIED Defense Technical Information Center Compilation Part Notice ADP012911 TITLE: Low Temperature Scanning Tunneling Spectroscopy of Different Individual Impurities on GaAs [110] Surface and in
More informationNovel determination of dierential-equation solutions: universal approximation method
Journal of Computational and Applied Mathematics 146 (2002) 443 457 www.elsevier.com/locate/cam Novel determination of dierential-equation solutions: universal approximation method Thananchai Leephakpreeda
More informationJ.F. Cari~nena structures and that of groupoid. So we will nd topological groupoids, Lie groupoids, symplectic groupoids, Poisson groupoids and so on.
Lie groupoids and algebroids in Classical and Quantum Mechanics 1 Jose F. Cari~nena Departamento de Fsica Teorica. Facultad de Ciencias. Universidad de Zaragoza, E-50009, Zaragoza, Spain. Abstract The
More informationMachine Learning Lecture 5
Machine Learning Lecture 5 Linear Discriminant Functions 26.10.2017 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Course Outline Fundamentals Bayes Decision Theory
More informationNonlinear Waves On the Geometric Algebra
Preprint Nonlinear Waves On the Geometric Algebra Carl Brannen (Dated: December, 004) Soliton solutions of nonlinear wave equations are postulated as a model for the elementary chiral fermions. The Dirac
More informationNONLINEAR CLASSIFICATION AND REGRESSION. J. Elder CSE 4404/5327 Introduction to Machine Learning and Pattern Recognition
NONLINEAR CLASSIFICATION AND REGRESSION Nonlinear Classification and Regression: Outline 2 Multi-Layer Perceptrons The Back-Propagation Learning Algorithm Generalized Linear Models Radial Basis Function
More informationoutput dimension input dimension Gaussian evidence Gaussian Gaussian evidence evidence from t +1 inputs and outputs at time t x t+2 x t-1 x t+1
To appear in M. S. Kearns, S. A. Solla, D. A. Cohn, (eds.) Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 999. Learning Nonlinear Dynamical Systems using an EM Algorithm Zoubin
More informationEcient Higher-order Neural Networks. for Classication and Function Approximation. Joydeep Ghosh and Yoan Shin. The University of Texas at Austin
Ecient Higher-order Neural Networks for Classication and Function Approximation Joydeep Ghosh and Yoan Shin Department of Electrical and Computer Engineering The University of Texas at Austin Austin, TX
More informationSeries Expansions for Analytic Systems Linear in the Controls
IEEE Decision and Control Conference Sydney, Australia, Dec 2 Series Expansions for Analytic Systems Linear in the Controls Francesco Bullo Coordinated Science Laboratory and General Engineering Department
More informationOn the construction of discrete gradients
On the construction of discrete gradients Elizabeth L. Mansfield School of Mathematics, Statistics and Actuarial Science, University of Kent, Canterbury, CT2 7NF, U.K. G. Reinout W. Quispel Department
More informationThe Hopf bifurcation with bounded noise
The Hopf bifurcation with bounded noise Ryan T. Botts 1, Ale Jan Homburg 2,3, and Todd R. Young 4 1 Department of Mathematical, Information & Computer Sciences, Point Loma Nazarene University, 3900 Lomaland
More informationStability and bifurcation of a simple neural network with multiple time delays.
Fields Institute Communications Volume, 999 Stability and bifurcation of a simple neural network with multiple time delays. Sue Ann Campbell Department of Applied Mathematics University of Waterloo Waterloo
More informationConvergence of The Multigrid Method With A Wavelet. Abstract. This new coarse grid operator is constructed using the wavelet
Convergence of The Multigrid Method With A Wavelet Coarse Grid Operator Bjorn Engquist Erding Luo y Abstract The convergence of the two-level multigrid method with a new coarse grid operator is studied.
More informationThe Erwin Schrodinger International Pasteurgasse 6/7. Institute for Mathematical Physics A-1090 Wien, Austria
ESI The Erwin Schrodinger International Pasteurgasse 6/7 Institute for Mathematical Physics A-1090 Wien, Austria On Twisted Tensor Products of Algebras Andreas Cap Hermann Schichl Jir Vanzura Vienna, Preprint
More informationMatrix factorization and minimal state space realization in the max-plus algebra
KULeuven Department of Electrical Engineering (ESAT) SISTA Technical report 96-69 Matrix factorization and minimal state space realization in the max-plus algebra B De Schutter and B De Moor If you want
More informationDiscrete time Generalized Cellular. Johan A.K. Suykens and Joos Vandewalle. Katholieke Universiteit Leuven
Discrete time Generalized Cellular Neural Networks within NL q Theory Johan A.K. Suykens and Joos Vandewalle Katholieke Universiteit Leuven Department of Electrical Engineering, ESAT-SISTA Kardinaal Mercierlaan
More informationAn introduction to Birkhoff normal form
An introduction to Birkhoff normal form Dario Bambusi Dipartimento di Matematica, Universitá di Milano via Saldini 50, 0133 Milano (Italy) 19.11.14 1 Introduction The aim of this note is to present an
More informationIMPROVED HIGH ORDER INTEGRATORS BASED ON MAGNUS EPANSION S. Blanes ;, F. Casas, and J. Ros 3 Abstract We build high order ecient numerical integration
UNIVERSITY OF CAMBRIDGE Numerical Analysis Reports Improved High Order Integrators Based on Magnus Expansion S. Blanes, F. Casas and J. Ros DAMTP 999/NA8 August, 999 Department of Applied Mathematics and
More informationIntroduction to symplectic integrator methods for solving diverse dynamical equations
Introduction to symplectic integrator methods for solving diverse dynamical equations Siu A. Chin Department of Physics and Astronomy Texas A&M University College Station, TX, 77840, USA Third Lecture
More information488 Solutions to the XOR Problem
488 Solutions to the XOR Problem Frans M. Coetzee * eoetzee@eee.emu.edu Department of Electrical Engineering Carnegie Mellon University Pittsburgh, PA 15213 Virginia L. Stonick ginny@eee.emu.edu Department
More informationNote that for a, b G, we have L a R b = R b L a, by the associative law. We'll use e to denote the identity element of G.
NOTES ON PRINCIPAL BUNDLES, CONNECTIONS, AND HOMOGENEOUS SPACES LANCE D. DRAGER 1. Introduction These notes are meant to accompany talks in the Geometry Seminar at Texas Tech during the Spring Semester
More informationArtificial Neural Networks. MGS Lecture 2
Artificial Neural Networks MGS 2018 - Lecture 2 OVERVIEW Biological Neural Networks Cell Topology: Input, Output, and Hidden Layers Functional description Cost functions Training ANNs Back-Propagation
More informationParallel layer perceptron
Neurocomputing 55 (2003) 771 778 www.elsevier.com/locate/neucom Letters Parallel layer perceptron Walmir M. Caminhas, Douglas A.G. Vieira, João A. Vasconcelos Department of Electrical Engineering, Federal
More informationAcid con- Exposure dose. Resist. Dissolution. rate. profile. centration. C(x,z) R(x,z) z(x) Photoresist. z Wafer
Optimal temperature proles for post-exposure bake of photo-resist Anders Hansson and Stephen Boyd Information Systems Laboratory Stanford University Stanford, CA 9435{95 ABSTRACT In this paper it is shown
More informationMatrix-valued functions
Matrix-valued functions Aim lecture: We solve some first order linear homogeneous differential equations using exponentials of matrices. Recall as in MATH2, the any function R M mn (C) : t A(t) can be
More information