CONTROL SYSTEMS, ROBOTICS AND AUTOMATION Vol. XI Control of Stochastic Systems - P.R. Kumar
|
|
- Merry Carmella Strickland
- 6 years ago
- Views:
Transcription
1 CONROL OF SOCHASIC SYSEMS P.R. Kumar Deparmen of Elecrical and Compuer Engineering, and Coordinaed Science Laboraory, Universiy of Illinois, Urbana-Champaign, USA. Keywords: Markov chains, ransiion probabiliies, conrolled Markov chains, noisy observaions, parially observed sysems, linear sochasic sysems, linear Gaussian sysems, conrolled auo regressive models, auoregressive moving average sysems wih exogenous inpus, cos funcions, ime-horizon, erminal cos, running cos, saefeedback policy, Markov policy, opimal cos-o-go, dynamic programming equaion, principle of opimaliy, linear quadraic Gaussian problem, equilibrium poins, sabiliy, counable Markov chains, seady-sae, super-maringales, Lyapunov funcions, sopping imes, recurrence, posiive recurrence, sochasic sabiliy, esimaion, Bayes Rules, Kalman filer, minimum mean-square error esimae, condiional mean, condiional covariance, poserior probabiliy disribuion, Bayesian and Non-Bayesian approaches o adapive conrol, parameer vecor, maximum likelihood, leas squares esimae, predicion error esimae, sysem idenificaion, consisency, separaion heorem, cerainy equivalence, informaion sae, hypersae, self-uning regulaors. Conens 1. Inroducion 2. Models of Sochasic Sysems 3. Opimal Sochasic Conrol 4. Sabiliy of Sochasic Sysems 5. Esimaion of Sochasic Sysems 6. Idenificaion and Parameer Esimaion of Sochasic Sysems 7. Conrol of Parially Observed Sysems 8. Adapive Conrol Glossary Bibliography Biographical Skech Summary We presen an accoun of several opics, modeling, conrol, esimaion, sabiliy, idenificaion and adapive conrol, which arise in he sudy of he conrol of sochasic sysems. 1. Inroducion A holisic reamen of he problem of conrol of sochasic sysems encompasses he following opics: (i) Models of sochasic sysems (ii) Opimal sochasic conrol
2 (iii) Sabiliy of sochasic sysems (iv) Esimaion of sochasic sysems (v) Idenificaion of sochasic sysems (vi) Conrol of parially observed sysems (vii) Adapive conrol We presen an ouline of each of hese opics which will enable he reader o obain an inegraed perspecive of he field. 2. Models of Sochasic Sysems A discree-ime sochasic process { x( )} = 0 is a Markov chain if p( x( + 1) x(0),, x( )) = p( x( + 1) x ( )). ha is, he condiional disribuion of he fuure sae x ( + 1) depends on he pas ( x(0),, x ( )) only hrough he presen sae x (). Indeed his jusifies he use of he name sae. he Markov chain can hen be described by is ransiion probabiliies p( x( + 1) x ( )). Exending his noion, one can describe a conrolled Markov chain by is conrolled ransiion probabiliies p ( x x, u ) which describe he condiional probabiliy of he nex sae x ( + 1) being x, when he curren sae x( ) = x, and an inpu u( ) = u is applied. If he sae x ( ) is no observed, hen i is common o model he observaions y ( ) by he condiional probabiliy disribuion p( yx ) which describes he probabiliy disribuion of he observaion y ( ) when he sae x( ) = x. + he sysem is hen called a parially observed conrolled Markov chain. If he ransiion probabiliies depend on he ime, hen one can describe he imevarying sysem by he pair of ransiion probabiliies p ( x x,u, ) and p( yx, ). A common deerminisic noise-free sae space model of a sysems in discree-ime is x( + 1) = f( x( ), u( ), ) y( ) = g( x( ), ), where x ( ) is he sae of he sysem a ime, u ( ) is he inpu applied a ime, and y () is he oupu a ime. he corresponding sochasic analog of he sae space model is
3 x( + 1) = f( x( ), u( ), w( ), ) y( ) = g( x( ), v( ) ), where w () is he noise enering he sae equaion, and v () is he noise enering he observaion equaion. hese noises are modeled as sochasic processes (see Models of Sochasic Sysems). If { w(0), w(1), w (2), } are muually independen, hen x ( ) is indeed he sae of a conrolled Markov chain. If furher { v(0), v(1), v (2), } are also muually independen, and w, v are independen of each oher, hen one has a parially observed conrolled Markov chain wrien in he form of sae and observaion equaions. If { w(0), w(1), w (2), } are no muually independen, hen one ofen models hem as he oupu of a sysem driven by independen random variables { n(0), n (1),, m(0), m(1), } z( + 1) = h( z( ), n( )) w( ) = k( z( ), m( )). In such siuaions, one can adjoin z o x and le ( x,z ) serve as he sae. A special and imporan case of such a sae space model is a linear sochasic sysem: x( + 1) = A( ) x( ) + B( ) u( ) + G( ) w( ) y() = C() x() + H() v(), where A(), B(), C(), G() and H () are ime-varying marices of appropriae dimensions. A model ha paricularly lends iself o analysis is when he noise processes w () and v () are joinly Gaussian sochasic processes (see Models of Sochasic Sysems). hen i is called a Linear-Gaussian model. Insead of dealing wih he sae x (), one can direcly model how he inpu influences he oupu, i.e., by an inpu-oupu model. he mos common model is a Conrol Auoregressive Moving Average Model (CARMA) or Auoregressive Moving Average Model wih Exogenous Inpus (ARMAX) model: y() + a1y( 1) + + any( n) = b0u() + b1u( 1) + + bu ( n) + w( ) + cw( 1) + + cw( n). n One can also consider he coninuous ime counerpar of he sae-space model: dx( ) = f( x( ), u( ), ) d+σ ( x( )) dw( ) dy() = g( x()) d+ dv(). 1 n
4 Here w ( ) and v ( ) are Brownian moion processes, and one has o inerpre he above sochasic differenial equaions in he appropriae mahemaical way. his requires a knowledge of Io sochasic inegrals and sochasic calculus. 3. Opimal Sochasic Conrol Consider he case of a discree-ime sochasic sysem where he sae x ( ) is direcly observed. How should one choose he conrol inpu { u ( )} o be applied o such a sysem? A common approach is o consider a cos funcion of he form E hx ( ( + 1)) + c( x( ), u ( )), = 0 and choose conrol inpus which minimize his expeced cos. Above is a ime horizon, hx ( ( + 1)) is he erminal cos, and c( x( ), u ( )) is he running cos. One minimizes his cos over he se of hisory dependen sraegies where u() = u( x(0),, x (),) is allowed o depend on he enire pas of he observaions and he curren ime. I can be shown ha wihin his class of hisory dependen sraegies one can resric aenion o sraegies of he form u() = u( x (),) where he inpu depends only on he curren sae and curren ime. Such a sraegy can be ermed as a sae feedback policy or a Markov policy. If one defines he opimal remaining cos or opimal cos-o-go from a sae x a ime by V( x, ): = Min u() E h( x( + 1)) + c( x( s), u( s)) x( ) = x, s= hen i can be shown ha his funcion saisfies he following equaion: V( x, ) = Min u c( ) + ( ) V ( + 1), x,u p x x, u x, x wih he erminal condiion V ( x, + 1) = h( x ). Essenially he above equaion says ha he opimal cos from a sae x a ime is obained by considering differen choices of an inpu u o apply a ime. For each such poenial inpu u, one deermines he curren cos c( x,u ) as well as he expeced cos from he sae reached a he nex ime insan. hen, one simply chooses he bes inpu o apply a he presen ime as he one which minimizes he sum of he expeced curren cos plus he expeced remaining cos. his equaion is called he dynamic programming equaion, and he logic leading o i as he principle of opimaliy. I also
5 follows ha if for ( x, ) one chooses he minimizing u, calling i ux, ( ), hen ux, ( ) is he opimal policy. hus he opimal policy can be chosen as a Markov or a sae feedback policy. he dynamic programming approach can be exended o oher models and siuaions, as shown in Dynamic Programming. A paricular special case of grea ineres in conrol is he so-called Linear-Quadraic- Gaussian (LQG) problem. For a linear sysem wih independen whie Gaussian noises w and v, x( + 1) = A( ) x( ) + B( ) u( ) + G( ) w( ) y() = C() x() + H() v(), one seeks o minimize a quadraic cos crierion: E x ( + 1) Sx( + 1) + ( x ( ) Q( ) x ( ) + u ( ) R( ) u ( )), = 1 where S 0, Q( ) 0 and R ( ) >0. he cos-o-go funcion urns ou o be quadraic funcion of he sae plus a deerminisic erm: V( x, ) = x S() x + γ (). By subsiuing his form in he dynamic programming equaion, one can solve for S ( ) and γ ( ) in erms of S+ ( 1) and γ ( + 1) (remember ha dynamic programming solves he problem backwards in ime). Wih he boundary condiions S ( + 1) = S and γ ( + 1) = 0, one hus obains recursions for S ( ) and γ ( ). From he minimizing argumen in he dynamic programming equaion one also deermines ha he opimal conrol law is of he form u() = K() x (), i.e., linear ime varying feedback, wih K ( ) expressible in erms of S. ( ) he LQG problem hus admis a clean soluion. he deails of he soluion are given in LQsochasic Conrol. Given ha he quadraic cos funcion is a reasonable crierion, and given he widespread usage of linear models, his soluion has proved o be eminenly useful in conrol sysem design. In many siuaions of ineres, e.g., in adapive conrol and self-uning regulaors, see Self-uning Conrol, one wishes o work wih inpu-oupu models wih quadraic coss. his is deal wih in Minimum Variance Conrol.
6 - - - O ACCESS ALL HE 18 PAGES OF HIS CHAPER, Click here Bibliography B. Anderson and J.B. Moore, Opimal Filering. Englewood Cliffs, NJ: Prenice-Hall, [Conains a comprehensive reamen of esimaion for linear sochasic sysems]. D.P. Bersekas, Dynamic Programming: Deerminisic and Sochasic Models. Englewood Cliffs, NJ: Prenice-Hall, [A comprehensive reamen of discree-ime dynamic programming]. G.C. Goodwin and K.S. Sin, Adapive Filering, Predicion and conrol. Englewood Cliffs, NJ: Prenice- Hall, [Conains a reamen of discree-ime modeling of linear sysems, as well as idenificaion and adapive conrol]. P.R. Kumar and P.P. Varaiya, Sochasic Sysems: Esimaion, Idenificaion and Adapive Conrol. Englewood Cliffs, NJ: Prenice-Hall, [Conains a concise reamen of several opics including modeling, conrol, esimaion, idenificaion, and adapaion]. H. Kushner, Inroducion o Sochasic Conrol. Hold, Rinehar and Winson, [Conains a reamen of sochasic conrol as well as sochasic sabiliy]. L.Ljung and. Södersrom, heory and Pracice of Recursive Idenificaion. Cambridge, MA: MI Press, [Conains a reamen of idenificaion and parameer esimaion for linear sysems]. Biographical Skech P.R. Kumar is he Franklin W. Woelge Professor of Elecrical and Compuer Engineering, and a Research Professor in he Coordinaed Science Laboraory, a he Universiy of Illinois, Urbana- Champaign. He was he recipien of he Donald P. Eckman Award of he American Auomaic Conrol Council. He has presened plenary lecures a he SIAM Annual Meeing and he SIAM Conrol Conference in 2001, he IEEE Conference on Decision and Conrol in San Anonio, exas, 1993, he SIAM Conference on Opimizaion in Chicago, 1992, he SIAM Annual Meeing a San Diego, 1994, Brazilian Auomaic Conrol Congress, and he hird Annual Semiconducor Manufacuring, Conrol and Opimizaion Workshop. He is co-auhor wih Pravin Varaiya of he book, Sochasic Sysems: Esimaion, Idenificaion and Adapive Conrol. He serves on he ediorial boards of Communicaions in Informaion and Sysems, Journal of Discree Even Dynamic Sysems; Mahemaics of Conrol Signals and Sysems; Mahemaical Problems in Engineering: Problems, heories and Applicaions; and in he pas has served as Associae Edior a Large of IEEE ransacions on Auomaic Conrol; Associae Edior of SIAM Journal on Conrol and Opimizaion; Sysems and Conrol Leers; Journal of Adapive Conrol and Signal Processing; and he IEEE ransacions on Auomaic Conrol. He is a Fellow of IEEE. Professor Kumar s curren research ineress are in wireless neworks, disribued real-ime sysems wafer fabricaion plans, and machine learning.
An Introduction to Backward Stochastic Differential Equations (BSDEs) PIMS Summer School 2016 in Mathematical Finance.
1 An Inroducion o Backward Sochasic Differenial Equaions (BSDEs) PIMS Summer School 2016 in Mahemaical Finance June 25, 2016 Chrisoph Frei cfrei@ualbera.ca This inroducion is based on Touzi [14], Bouchard
More informationGeorey E. Hinton. University oftoronto. Technical Report CRG-TR February 22, Abstract
Parameer Esimaion for Linear Dynamical Sysems Zoubin Ghahramani Georey E. Hinon Deparmen of Compuer Science Universiy oftorono 6 King's College Road Torono, Canada M5S A4 Email: zoubin@cs.orono.edu Technical
More informationRecursive Least-Squares Fixed-Interval Smoother Using Covariance Information based on Innovation Approach in Linear Continuous Stochastic Systems
8 Froniers in Signal Processing, Vol. 1, No. 1, July 217 hps://dx.doi.org/1.2266/fsp.217.112 Recursive Leas-Squares Fixed-Inerval Smooher Using Covariance Informaion based on Innovaion Approach in Linear
More informationNotes on Kalman Filtering
Noes on Kalman Filering Brian Borchers and Rick Aser November 7, Inroducion Daa Assimilaion is he problem of merging model predicions wih acual measuremens of a sysem o produce an opimal esimae of he curren
More informationAugmented Reality II - Kalman Filters - Gudrun Klinker May 25, 2004
Augmened Realiy II Kalman Filers Gudrun Klinker May 25, 2004 Ouline Moivaion Discree Kalman Filer Modeled Process Compuing Model Parameers Algorihm Exended Kalman Filer Kalman Filer for Sensor Fusion Lieraure
More informationSZG Macro 2011 Lecture 3: Dynamic Programming. SZG macro 2011 lecture 3 1
SZG Macro 2011 Lecure 3: Dynamic Programming SZG macro 2011 lecure 3 1 Background Our previous discussion of opimal consumpion over ime and of opimal capial accumulaion sugges sudying he general decision
More informationOn the Separation Theorem of Stochastic Systems in the Case Of Continuous Observation Channels with Memory
Journal of Physics: Conference eries PAPER OPEN ACCE On he eparaion heorem of ochasic ysems in he Case Of Coninuous Observaion Channels wih Memory o cie his aricle: V Rozhova e al 15 J. Phys.: Conf. er.
More information( ) ( ) if t = t. It must satisfy the identity. So, bulkiness of the unit impulse (hyper)function is equal to 1. The defining characteristic is
UNIT IMPULSE RESPONSE, UNIT STEP RESPONSE, STABILITY. Uni impulse funcion (Dirac dela funcion, dela funcion) rigorously defined is no sricly a funcion, bu disribuion (or measure), precise reamen requires
More informationAn Introduction to Malliavin calculus and its applications
An Inroducion o Malliavin calculus and is applicaions Lecure 5: Smoohness of he densiy and Hörmander s heorem David Nualar Deparmen of Mahemaics Kansas Universiy Universiy of Wyoming Summer School 214
More informationSignaling equilibria for dynamic LQG games with. asymmetric information
Signaling equilibria for dynamic LQG games wih asymmeric informaion Deepanshu Vasal and Achilleas Anasasopoulos Absrac We consider a finie horizon dynamic game wih wo players who observe heir ypes privaely
More informationSliding Mode Extremum Seeking Control for Linear Quadratic Dynamic Game
Sliding Mode Exremum Seeking Conrol for Linear Quadraic Dynamic Game Yaodong Pan and Ümi Özgüner ITS Research Group, AIST Tsukuba Eas Namiki --, Tsukuba-shi,Ibaraki-ken 5-856, Japan e-mail: pan.yaodong@ais.go.jp
More informationBU Macro BU Macro Fall 2008, Lecture 4
Dynamic Programming BU Macro 2008 Lecure 4 1 Ouline 1. Cerainy opimizaion problem used o illusrae: a. Resricions on exogenous variables b. Value funcion c. Policy funcion d. The Bellman equaion and an
More informationState-Space Models. Initialization, Estimation and Smoothing of the Kalman Filter
Sae-Space Models Iniializaion, Esimaion and Smoohing of he Kalman Filer Iniializaion of he Kalman Filer The Kalman filer shows how o updae pas predicors and he corresponding predicion error variances when
More informationMean-square Stability Control for Networked Systems with Stochastic Time Delay
JOURNAL OF SIMULAION VOL. 5 NO. May 7 Mean-square Sabiliy Conrol for Newored Sysems wih Sochasic ime Delay YAO Hejun YUAN Fushun School of Mahemaics and Saisics Anyang Normal Universiy Anyang Henan. 455
More informationSUFFICIENT CONDITIONS FOR EXISTENCE SOLUTION OF LINEAR TWO-POINT BOUNDARY PROBLEM IN MINIMIZATION OF QUADRATIC FUNCTIONAL
HE PUBLISHING HOUSE PROCEEDINGS OF HE ROMANIAN ACADEMY, Series A, OF HE ROMANIAN ACADEMY Volume, Number 4/200, pp 287 293 SUFFICIEN CONDIIONS FOR EXISENCE SOLUION OF LINEAR WO-POIN BOUNDARY PROBLEM IN
More informationLecture 20: Riccati Equations and Least Squares Feedback Control
34-5 LINEAR SYSTEMS Lecure : Riccai Equaions and Leas Squares Feedback Conrol 5.6.4 Sae Feedback via Riccai Equaions A recursive approach in generaing he marix-valued funcion W ( ) equaion for i for he
More informationZürich. ETH Master Course: L Autonomous Mobile Robots Localization II
Roland Siegwar Margaria Chli Paul Furgale Marco Huer Marin Rufli Davide Scaramuzza ETH Maser Course: 151-0854-00L Auonomous Mobile Robos Localizaion II ACT and SEE For all do, (predicion updae / ACT),
More informationAnno accademico 2006/2007. Davide Migliore
Roboica Anno accademico 2006/2007 Davide Migliore migliore@ele.polimi.i Today Eercise session: An Off-side roblem Robo Vision Task Measuring NBA layers erformance robabilisic Roboics Inroducion The Bayesian
More informationInstitute for Mathematical Methods in Economics. University of Technology Vienna. Singapore, May Manfred Deistler
MULTIVARIATE TIME SERIES ANALYSIS AND FORECASTING Manfred Deisler E O S Economerics and Sysems Theory Insiue for Mahemaical Mehods in Economics Universiy of Technology Vienna Singapore, May 2004 Inroducion
More informationExcel-Based Solution Method For The Optimal Policy Of The Hadley And Whittin s Exact Model With Arma Demand
Excel-Based Soluion Mehod For The Opimal Policy Of The Hadley And Whiin s Exac Model Wih Arma Demand Kal Nami School of Business and Economics Winson Salem Sae Universiy Winson Salem, NC 27110 Phone: (336)750-2338
More informationRecursive Estimation and Identification of Time-Varying Long- Term Fading Channels
Recursive Esimaion and Idenificaion of ime-varying Long- erm Fading Channels Mohammed M. Olama, Kiran K. Jaladhi, Seddi M. Djouadi, and Charalambos D. Charalambous 2 Universiy of ennessee Deparmen of Elecrical
More informationTwo Popular Bayesian Estimators: Particle and Kalman Filters. McGill COMP 765 Sept 14 th, 2017
Two Popular Bayesian Esimaors: Paricle and Kalman Filers McGill COMP 765 Sep 14 h, 2017 1 1 1, dx x Bel x u x P x z P Recall: Bayes Filers,,,,,,, 1 1 1 1 u z u x P u z u x z P Bayes z = observaion u =
More informationarxiv: v1 [math.ca] 15 Nov 2016
arxiv:6.599v [mah.ca] 5 Nov 26 Counerexamples on Jumarie s hree basic fracional calculus formulae for non-differeniable coninuous funcions Cheng-shi Liu Deparmen of Mahemaics Norheas Peroleum Universiy
More informationApplications in Industry (Extended) Kalman Filter. Week Date Lecture Title
hp://elec34.com Applicaions in Indusry (Eended) Kalman Filer 26 School of Informaion echnology and Elecrical Engineering a he Universiy of Queensland Lecure Schedule: Week Dae Lecure ile 29-Feb Inroducion
More informationACE 562 Fall Lecture 4: Simple Linear Regression Model: Specification and Estimation. by Professor Scott H. Irwin
ACE 56 Fall 005 Lecure 4: Simple Linear Regression Model: Specificaion and Esimaion by Professor Sco H. Irwin Required Reading: Griffihs, Hill and Judge. "Simple Regression: Economic and Saisical Model
More informationAn introduction to the theory of SDDP algorithm
An inroducion o he heory of SDDP algorihm V. Leclère (ENPC) Augus 1, 2014 V. Leclère Inroducion o SDDP Augus 1, 2014 1 / 21 Inroducion Large scale sochasic problem are hard o solve. Two ways of aacking
More informationDiebold, Chapter 7. Francis X. Diebold, Elements of Forecasting, 4th Edition (Mason, Ohio: Cengage Learning, 2006). Chapter 7. Characterizing Cycles
Diebold, Chaper 7 Francis X. Diebold, Elemens of Forecasing, 4h Ediion (Mason, Ohio: Cengage Learning, 006). Chaper 7. Characerizing Cycles Afer compleing his reading you should be able o: Define covariance
More informationReferences are appeared in the last slide. Last update: (1393/08/19)
SYSEM IDEIFICAIO Ali Karimpour Associae Professor Ferdowsi Universi of Mashhad References are appeared in he las slide. Las updae: 0..204 393/08/9 Lecure 5 lecure 5 Parameer Esimaion Mehods opics o be
More informationModeling Economic Time Series with Stochastic Linear Difference Equations
A. Thiemer, SLDG.mcd, 6..6 FH-Kiel Universiy of Applied Sciences Prof. Dr. Andreas Thiemer e-mail: andreas.hiemer@fh-kiel.de Modeling Economic Time Series wih Sochasic Linear Difference Equaions Summary:
More informationChapter 8 The Complete Response of RL and RC Circuits
Chaper 8 The Complee Response of RL and RC Circuis Seoul Naional Universiy Deparmen of Elecrical and Compuer Engineering Wha is Firs Order Circuis? Circuis ha conain only one inducor or only one capacior
More informationThe Optimal Stopping Time for Selling an Asset When It Is Uncertain Whether the Price Process Is Increasing or Decreasing When the Horizon Is Infinite
American Journal of Operaions Research, 08, 8, 8-9 hp://wwwscirporg/journal/ajor ISSN Online: 60-8849 ISSN Prin: 60-8830 The Opimal Sopping Time for Selling an Asse When I Is Uncerain Wheher he Price Process
More informationApplication of a Stochastic-Fuzzy Approach to Modeling Optimal Discrete Time Dynamical Systems by Using Large Scale Data Processing
Applicaion of a Sochasic-Fuzzy Approach o Modeling Opimal Discree Time Dynamical Sysems by Using Large Scale Daa Processing AA WALASZE-BABISZEWSA Deparmen of Compuer Engineering Opole Universiy of Technology
More informationSignal and System (Chapter 3. Continuous-Time Systems)
Signal and Sysem (Chaper 3. Coninuous-Time Sysems) Prof. Kwang-Chun Ho kwangho@hansung.ac.kr Tel: 0-760-453 Fax:0-760-4435 1 Dep. Elecronics and Informaion Eng. 1 Nodes, Branches, Loops A nework wih b
More informationTime series model fitting via Kalman smoothing and EM estimation in TimeModels.jl
Time series model fiing via Kalman smoohing and EM esimaion in TimeModels.jl Gord Sephen Las updaed: January 206 Conens Inroducion 2. Moivaion and Acknowledgemens....................... 2.2 Noaion......................................
More information3.1.3 INTRODUCTION TO DYNAMIC OPTIMIZATION: DISCRETE TIME PROBLEMS. A. The Hamiltonian and First-Order Conditions in a Finite Time Horizon
3..3 INRODUCION O DYNAMIC OPIMIZAION: DISCREE IME PROBLEMS A. he Hamilonian and Firs-Order Condiions in a Finie ime Horizon Define a new funcion, he Hamilonian funcion, H. H he change in he oal value of
More informationDecentralized Stochastic Control with Partial History Sharing: A Common Information Approach
1 Decenralized Sochasic Conrol wih Parial Hisory Sharing: A Common Informaion Approach Ashuosh Nayyar, Adiya Mahajan and Demoshenis Tenekezis arxiv:1209.1695v1 [cs.sy] 8 Sep 2012 Absrac A general model
More information12: AUTOREGRESSIVE AND MOVING AVERAGE PROCESSES IN DISCRETE TIME. Σ j =
1: AUTOREGRESSIVE AND MOVING AVERAGE PROCESSES IN DISCRETE TIME Moving Averages Recall ha a whie noise process is a series { } = having variance σ. The whie noise process has specral densiy f (λ) = of
More informationACE 562 Fall Lecture 5: The Simple Linear Regression Model: Sampling Properties of the Least Squares Estimators. by Professor Scott H.
ACE 56 Fall 005 Lecure 5: he Simple Linear Regression Model: Sampling Properies of he Leas Squares Esimaors by Professor Sco H. Irwin Required Reading: Griffihs, Hill and Judge. "Inference in he Simple
More information2.160 System Identification, Estimation, and Learning. Lecture Notes No. 8. March 6, 2006
2.160 Sysem Idenificaion, Esimaion, and Learning Lecure Noes No. 8 March 6, 2006 4.9 Eended Kalman Filer In many pracical problems, he process dynamics are nonlinear. w Process Dynamics v y u Model (Linearized)
More informationLECTURE 1: GENERALIZED RAY KNIGHT THEOREM FOR FINITE MARKOV CHAINS
LECTURE : GENERALIZED RAY KNIGHT THEOREM FOR FINITE MARKOV CHAINS We will work wih a coninuous ime reversible Markov chain X on a finie conneced sae space, wih generaor Lf(x = y q x,yf(y. (Recall ha q
More informationOptimal Investment under Dynamic Risk Constraints and Partial Information
Opimal Invesmen under Dynamic Risk Consrains and Parial Informaion Wolfgang Puschögl Johann Radon Insiue for Compuaional and Applied Mahemaics (RICAM) Ausrian Academy of Sciences www.ricam.oeaw.ac.a 2
More information6. Stochastic calculus with jump processes
A) Trading sraegies (1/3) Marke wih d asses S = (S 1,, S d ) A rading sraegy can be modelled wih a vecor φ describing he quaniies invesed in each asse a each insan : φ = (φ 1,, φ d ) The value a of a porfolio
More informationHomogenization of random Hamilton Jacobi Bellman Equations
Probabiliy, Geomery and Inegrable Sysems MSRI Publicaions Volume 55, 28 Homogenizaion of random Hamilon Jacobi Bellman Equaions S. R. SRINIVASA VARADHAN ABSTRACT. We consider nonlinear parabolic equaions
More informationCash Flow Valuation Mode Lin Discrete Time
IOSR Journal of Mahemaics (IOSR-JM) e-issn: 2278-5728,p-ISSN: 2319-765X, 6, Issue 6 (May. - Jun. 2013), PP 35-41 Cash Flow Valuaion Mode Lin Discree Time Olayiwola. M. A. and Oni, N. O. Deparmen of Mahemaics
More informationL07. KALMAN FILTERING FOR NON-LINEAR SYSTEMS. NA568 Mobile Robotics: Methods & Algorithms
L07. KALMAN FILTERING FOR NON-LINEAR SYSTEMS NA568 Mobile Roboics: Mehods & Algorihms Today s Topic Quick review on (Linear) Kalman Filer Kalman Filering for Non-Linear Sysems Exended Kalman Filer (EKF)
More informationVehicle Arrival Models : Headway
Chaper 12 Vehicle Arrival Models : Headway 12.1 Inroducion Modelling arrival of vehicle a secion of road is an imporan sep in raffic flow modelling. I has imporan applicaion in raffic flow simulaion where
More informationOptimal Path Planning for Flexible Redundant Robot Manipulators
25 WSEAS In. Conf. on DYNAMICAL SYSEMS and CONROL, Venice, Ialy, November 2-4, 25 (pp363-368) Opimal Pah Planning for Flexible Redundan Robo Manipulaors H. HOMAEI, M. KESHMIRI Deparmen of Mechanical Engineering
More informationTHE LQG control problem [1] concerns linear systems
Orhogonal Funcions Approach o LQG Conrol B M Mohan Senior Member, IEEE, and Sanjeeb Kumar Kar Absrac In his paper a unified approach via block-pulse funcions (BPFs) or shifed Legendre polynomials (SLPs)
More informationFractional Method of Characteristics for Fractional Partial Differential Equations
Fracional Mehod of Characerisics for Fracional Parial Differenial Equaions Guo-cheng Wu* Modern Teile Insiue, Donghua Universiy, 188 Yan-an ilu Road, Shanghai 51, PR China Absrac The mehod of characerisics
More information0.1 MAXIMUM LIKELIHOOD ESTIMATION EXPLAINED
0.1 MAXIMUM LIKELIHOOD ESTIMATIO EXPLAIED Maximum likelihood esimaion is a bes-fi saisical mehod for he esimaion of he values of he parameers of a sysem, based on a se of observaions of a random variable
More information1. An introduction to dynamic optimization -- Optimal Control and Dynamic Programming AGEC
This documen was generaed a :45 PM 8/8/04 Copyrigh 04 Richard T. Woodward. An inroducion o dynamic opimizaion -- Opimal Conrol and Dynamic Programming AGEC 637-04 I. Overview of opimizaion Opimizaion is
More information10. State Space Methods
. Sae Space Mehods. Inroducion Sae space modelling was briefly inroduced in chaper. Here more coverage is provided of sae space mehods before some of heir uses in conrol sysem design are covered in he
More informationStability and Bifurcation in a Neural Network Model with Two Delays
Inernaional Mahemaical Forum, Vol. 6, 11, no. 35, 175-1731 Sabiliy and Bifurcaion in a Neural Nework Model wih Two Delays GuangPing Hu and XiaoLing Li School of Mahemaics and Physics, Nanjing Universiy
More informationPhysics 235 Chapter 2. Chapter 2 Newtonian Mechanics Single Particle
Chaper 2 Newonian Mechanics Single Paricle In his Chaper we will review wha Newon s laws of mechanics ell us abou he moion of a single paricle. Newon s laws are only valid in suiable reference frames,
More informationLecture 1 Overview. course mechanics. outline & topics. what is a linear dynamical system? why study linear systems? some examples
EE263 Auumn 27-8 Sephen Boyd Lecure 1 Overview course mechanics ouline & opics wha is a linear dynamical sysem? why sudy linear sysems? some examples 1 1 Course mechanics all class info, lecures, homeworks,
More informationGMM - Generalized Method of Moments
GMM - Generalized Mehod of Momens Conens GMM esimaion, shor inroducion 2 GMM inuiion: Maching momens 2 3 General overview of GMM esimaion. 3 3. Weighing marix...........................................
More informationRANDOM LAGRANGE MULTIPLIERS AND TRANSVERSALITY
ECO 504 Spring 2006 Chris Sims RANDOM LAGRANGE MULTIPLIERS AND TRANSVERSALITY 1. INTRODUCTION Lagrange muliplier mehods are sandard fare in elemenary calculus courses, and hey play a cenral role in economic
More informationdy dx = xey (a) y(0) = 2 (b) y(1) = 2.5 SOLUTION: See next page
Assignmen 1 MATH 2270 SOLUTION Please wrie ou complee soluions for each of he following 6 problems (one more will sill be added). You may, of course, consul wih your classmaes, he exbook or oher resources,
More informationLAPLACE TRANSFORM AND TRANSFER FUNCTION
CHBE320 LECTURE V LAPLACE TRANSFORM AND TRANSFER FUNCTION Professor Dae Ryook Yang Spring 2018 Dep. of Chemical and Biological Engineering 5-1 Road Map of he Lecure V Laplace Transform and Transfer funcions
More informationOnline Appendix to Solution Methods for Models with Rare Disasters
Online Appendix o Soluion Mehods for Models wih Rare Disasers Jesús Fernández-Villaverde and Oren Levinal In his Online Appendix, we presen he Euler condiions of he model, we develop he pricing Calvo block,
More informationEssential Microeconomics : OPTIMAL CONTROL 1. Consider the following class of optimization problems
Essenial Microeconomics -- 6.5: OPIMAL CONROL Consider he following class of opimizaion problems Max{ U( k, x) + U+ ( k+ ) k+ k F( k, x)}. { x, k+ } = In he language of conrol heory, he vecor k is he vecor
More informationIntroduction to Probability and Statistics Slides 4 Chapter 4
Inroducion o Probabiliy and Saisics Slides 4 Chaper 4 Ammar M. Sarhan, asarhan@mahsa.dal.ca Deparmen of Mahemaics and Saisics, Dalhousie Universiy Fall Semeser 8 Dr. Ammar Sarhan Chaper 4 Coninuous Random
More informationThis document was generated at 1:04 PM, 09/10/13 Copyright 2013 Richard T. Woodward. 4. End points and transversality conditions AGEC
his documen was generaed a 1:4 PM, 9/1/13 Copyrigh 213 Richard. Woodward 4. End poins and ransversaliy condiions AGEC 637-213 F z d Recall from Lecure 3 ha a ypical opimal conrol problem is o maimize (,,
More informationDeep Learning: Theory, Techniques & Applications - Recurrent Neural Networks -
Deep Learning: Theory, Techniques & Applicaions - Recurren Neural Neworks - Prof. Maeo Maeucci maeo.maeucci@polimi.i Deparmen of Elecronics, Informaion and Bioengineering Arificial Inelligence and Roboics
More informationR.#W.#Erickson# Department#of#Electrical,#Computer,#and#Energy#Engineering# University#of#Colorado,#Boulder#
.#W.#Erickson# Deparmen#of#Elecrical,#Compuer,#and#Energy#Engineering# Universiy#of#Colorado,#Boulder# Chaper 2 Principles of Seady-Sae Converer Analysis 2.1. Inroducion 2.2. Inducor vol-second balance,
More information( ) = Q 0. ( ) R = R dq. ( t) = I t
ircuis onceps The addiion of a simple capacior o a circui of resisors allows wo relaed phenomena o occur The observaion ha he ime-dependence of a complex waveform is alered by he circui is referred o as
More informationA variational radial basis function approximation for diffusion processes.
A variaional radial basis funcion approximaion for diffusion processes. Michail D. Vreas, Dan Cornford and Yuan Shen {vreasm, d.cornford, y.shen}@ason.ac.uk Ason Universiy, Birmingham, UK hp://www.ncrg.ason.ac.uk
More informationSTATE-SPACE MODELLING. A mass balance across the tank gives:
B. Lennox and N.F. Thornhill, 9, Sae Space Modelling, IChemE Process Managemen and Conrol Subjec Group Newsleer STE-SPACE MODELLING Inroducion: Over he pas decade or so here has been an ever increasing
More informationSystem of Linear Differential Equations
Sysem of Linear Differenial Equaions In "Ordinary Differenial Equaions" we've learned how o solve a differenial equaion for a variable, such as: y'k5$e K2$x =0 solve DE yx = K 5 2 ek2 x C_C1 2$y''C7$y
More informationThe Strong Law of Large Numbers
Lecure 9 The Srong Law of Large Numbers Reading: Grimme-Sirzaker 7.2; David Williams Probabiliy wih Maringales 7.2 Furher reading: Grimme-Sirzaker 7.1, 7.3-7.5 Wih he Convergence Theorem (Theorem 54) and
More informationA DELAY-DEPENDENT STABILITY CRITERIA FOR T-S FUZZY SYSTEM WITH TIME-DELAYS
A DELAY-DEPENDENT STABILITY CRITERIA FOR T-S FUZZY SYSTEM WITH TIME-DELAYS Xinping Guan ;1 Fenglei Li Cailian Chen Insiue of Elecrical Engineering, Yanshan Universiy, Qinhuangdao, 066004, China. Deparmen
More informationSubway stations energy and air quality management
Subway saions energy and air qualiy managemen wih sochasic opimizaion Trisan Rigau 1,2,4, Advisors: P. Carpenier 3, J.-Ph. Chancelier 2, M. De Lara 2 EFFICACITY 1 CERMICS, ENPC 2 UMA, ENSTA 3 LISIS, IFSTTAR
More informationSEIF, EnKF, EKF SLAM. Pieter Abbeel UC Berkeley EECS
SEIF, EnKF, EKF SLAM Pieer Abbeel UC Berkeley EECS Informaion Filer From an analyical poin of view == Kalman filer Difference: keep rack of he inverse covariance raher han he covariance marix [maer of
More informationKalman Bucy filtering equations of forward and backward stochastic systems and applications to recursive optimal control problems
J. Mah. Anal. Appl. 34 8) 18 196 www.elsevier.com/locae/jmaa Kalman Bucy filering equaions of forward and backward sochasic sysems and applicaions o recursive opimal conrol problems Guangchen Wang a,b,,zhenwu
More informationd 1 = c 1 b 2 - b 1 c 2 d 2 = c 1 b 3 - b 1 c 3
and d = c b - b c c d = c b - b c c This process is coninued unil he nh row has been compleed. The complee array of coefficiens is riangular. Noe ha in developing he array an enire row may be divided or
More informationMean-Variance Hedging for General Claims
Projekbereich B Discussion Paper No. B 167 Mean-Variance Hedging for General Claims by Marin Schweizer ) Ocober 199 ) Financial suppor by Deusche Forschungsgemeinschaf, Sonderforschungsbereich 33 a he
More informationChapter 2. First Order Scalar Equations
Chaper. Firs Order Scalar Equaions We sar our sudy of differenial equaions in he same way he pioneers in his field did. We show paricular echniques o solve paricular ypes of firs order differenial equaions.
More informationAnti-Disturbance Control for Multiple Disturbances
Workshop a 3 ACC Ani-Disurbance Conrol for Muliple Disurbances Lei Guo (lguo@buaa.edu.cn) Naional Key Laboraory on Science and Technology on Aircraf Conrol, Beihang Universiy, Beijing, 9, P.R. China. Presened
More informationHidden Markov Models. Adapted from. Dr Catherine Sweeney-Reed s slides
Hidden Markov Models Adaped from Dr Caherine Sweeney-Reed s slides Summary Inroducion Descripion Cenral in HMM modelling Exensions Demonsraion Specificaion of an HMM Descripion N - number of saes Q = {q
More information1. An introduction to dynamic optimization -- Optimal Control and Dynamic Programming AGEC
This documen was generaed a :37 PM, 1/11/018 Copyrigh 018 Richard T. Woodward 1. An inroducion o dynamic opimiaion -- Opimal Conrol and Dynamic Programming AGEC 64-018 I. Overview of opimiaion Opimiaion
More informationChapter 7 Response of First-order RL and RC Circuits
Chaper 7 Response of Firs-order RL and RC Circuis 7.- The Naural Response of RL and RC Circuis 7.3 The Sep Response of RL and RC Circuis 7.4 A General Soluion for Sep and Naural Responses 7.5 Sequenial
More informationThe expectation value of the field operator.
The expecaion value of he field operaor. Dan Solomon Universiy of Illinois Chicago, IL dsolom@uic.edu June, 04 Absrac. Much of he mahemaical developmen of quanum field heory has been in suppor of deermining
More informationStructural results for partially nested LQG systems over graphs
Srucural resuls for parially nesed LQG sysems over graphs Ashuosh Nayyar 1 Lauren Lessard 2 American Conrol Conference, pp. 5457 5464, 2015 Absrac We idenify a broad class of decenralized oupufeedback
More informationTracking. Announcements
Tracking Tuesday, Nov 24 Krisen Grauman UT Ausin Announcemens Pse 5 ou onigh, due 12/4 Shorer assignmen Auo exension il 12/8 I will no hold office hours omorrow 5 6 pm due o Thanksgiving 1 Las ime: Moion
More informationOptimal Investment Strategy Insurance Company
Opimal Invesmen Sraegy for a Non-Life Insurance Company Łukasz Delong Warsaw School of Economics Insiue of Economerics Division of Probabilisic Mehods Probabiliy space Ω I P F I I I he filraion saisfies
More informationStable approximations of optimal filters
Sable approximaions of opimal filers Joaquin Miguez Deparmen of Signal Theory & Communicaions, Universidad Carlos III de Madrid. E-mail: joaquin.miguez@uc3m.es Join work wih Dan Crisan (Imperial College
More informationOn a Fractional Stochastic Landau-Ginzburg Equation
Applied Mahemaical Sciences, Vol. 4, 1, no. 7, 317-35 On a Fracional Sochasic Landau-Ginzburg Equaion Nguyen Tien Dung Deparmen of Mahemaics, FPT Universiy 15B Pham Hung Sree, Hanoi, Vienam dungn@fp.edu.vn
More informationHidden Markov Models
Hidden Markov Models Probabilisic reasoning over ime So far, we ve mosly deal wih episodic environmens Excepions: games wih muliple moves, planning In paricular, he Bayesian neworks we ve seen so far describe
More informationR t. C t P t. + u t. C t = αp t + βr t + v t. + β + w t
Exercise 7 C P = α + β R P + u C = αp + βr + v (a) (b) C R = α P R + β + w (c) Assumpions abou he disurbances u, v, w : Classical assumions on he disurbance of one of he equaions, eg. on (b): E(v v s P,
More informationBlock Diagram of a DCS in 411
Informaion source Forma A/D From oher sources Pulse modu. Muliplex Bandpass modu. X M h: channel impulse response m i g i s i Digial inpu Digial oupu iming and synchronizaion Digial baseband/ bandpass
More information2.4 Cuk converter example
2.4 Cuk converer example C 1 Cuk converer, wih ideal swich i 1 i v 1 2 1 2 C 2 v 2 Cuk converer: pracical realizaion using MOSFET and diode C 1 i 1 i v 1 2 Q 1 D 1 C 2 v 2 28 Analysis sraegy This converer
More informationA Bayesian Approach to Spectral Analysis
Chirped Signals A Bayesian Approach o Specral Analysis Chirped signals are oscillaing signals wih ime variable frequencies, usually wih a linear variaion of frequency wih ime. E.g. f() = A cos(ω + α 2
More informationThe Potential Effectiveness of the Detection of Pulsed Signals in the Non-Uniform Sampling
The Poenial Effeciveness of he Deecion of Pulsed Signals in he Non-Uniform Sampling Arhur Smirnov, Sanislav Vorobiev and Ajih Abraham 3, 4 Deparmen of Compuer Science, Universiy of Illinois a Chicago,
More informationTesting for a Single Factor Model in the Multivariate State Space Framework
esing for a Single Facor Model in he Mulivariae Sae Space Framework Chen C.-Y. M. Chiba and M. Kobayashi Inernaional Graduae School of Social Sciences Yokohama Naional Universiy Japan Faculy of Economics
More informationnot to be republished NCERT MATHEMATICAL MODELLING Appendix 2 A.2.1 Introduction A.2.2 Why Mathematical Modelling?
256 MATHEMATICS A.2.1 Inroducion In class XI, we have learn abou mahemaical modelling as an aemp o sudy some par (or form) of some real-life problems in mahemaical erms, i.e., he conversion of a physical
More informationExamples of Dynamic Programming Problems
M.I.T. 5.450-Fall 00 Sloan School of Managemen Professor Leonid Kogan Examples of Dynamic Programming Problems Problem A given quaniy X of a single resource is o be allocaed opimally among N producion
More informationCHBE320 LECTURE IV MATHEMATICAL MODELING OF CHEMICAL PROCESS. Professor Dae Ryook Yang
CHBE320 LECTURE IV MATHEMATICAL MODELING OF CHEMICAL PROCESS Professor Dae Ryook Yang Spring 208 Dep. of Chemical and Biological Engineering CHBE320 Process Dynamics and Conrol 4- Road Map of he Lecure
More informationElements of Stochastic Processes Lecture II Hamid R. Rabiee
Sochasic Processes Elemens of Sochasic Processes Lecure II Hamid R. Rabiee Overview Reading Assignmen Chaper 9 of exbook Furher Resources MIT Open Course Ware S. Karlin and H. M. Taylor, A Firs Course
More informationThe Asymptotic Behavior of Nonoscillatory Solutions of Some Nonlinear Dynamic Equations on Time Scales
Advances in Dynamical Sysems and Applicaions. ISSN 0973-5321 Volume 1 Number 1 (2006, pp. 103 112 c Research India Publicaions hp://www.ripublicaion.com/adsa.hm The Asympoic Behavior of Nonoscillaory Soluions
More informationMartingales Stopping Time Processes
IOSR Journal of Mahemaics (IOSR-JM) e-issn: 2278-5728, p-issn: 2319-765. Volume 11, Issue 1 Ver. II (Jan - Feb. 2015), PP 59-64 www.iosrjournals.org Maringales Sopping Time Processes I. Fulaan Deparmen
More information