Optimal Control of Linear Systems with Stochastic Parameters for Variance Suppression: The Finite Time Case
|
|
- Erika Byrd
- 5 years ago
- Views:
Transcription
1 Optimal Control of Linear Systems with Stochastic Parameters for Variance Suppression: The inite Time Case Kenji ujimoto Soraki Ogawa Yuhei Ota Makishi Nakayama Nagoya University, Department of Mechanical Science and Engineering, Nagoya , JAPAN Kobe Steel, LTD., Kobe , JAPAN Abstract: In this paper, we consider an optimal control problem for a linear discrete time system with stochastic parameters. Whereas traditional stochastic optimal control theory only treats systems with deterministic parameters with stochastic noises, this paper focuses on systems with both stochastic parameters and stochastic noises. We derive an optimal control law for a novel cost function by which the designer can adjust the trade-off between the average and the variance of the states. urthermore, a numerical simulation shows the effectiveness of the proposed method. Keywords: Stochastic optimal control problems; Probabilistic robustness; Linear systems 1. INTRODUCTION Stochastic and statistical methods are used in several problems in control systems theory. In particular, system identification and estimation methods rely on them. Recently, it is reported that Bayesian estimation and the related estimation tools in machine learning Bishop 26 are applied to system identification of state-space models and statistical information of the system parameter become available for control Barber and Chiappa 27; ukunaga et al. 28. There is another paper reporting that the quality of the products are estimated by those methods in process control Kano 21. So called randomized approach is also proposed to apply statistical tools to controller design problems Calafiore et al. 24. On the other hand, optimal control is an important and established control method. LQG Linear Quadratic Gaussian method can take care of optimal control problems with stochastic disturbances Aström 26; Mendel and Gieseking Stochastic control theory has been developed by many authors based on it which mainly focuses on systems with deterministic parameters and stochastic noises. In stochastic control theory, MCV Minimum Cost Variance control Sain 1966; Sain et al and RS Risk Sensitive control Whittle 1981 were proposed which can suppress the variance of the cost function with respect to the stochastic disturbance. Those existing results focus on the systems with deterministic parameters whereas they can take care of stochastic disturbances. In order to utilize the Bayesian estimation results for state space systems Barber and Chiappa 27; ukunaga et al. 28, controller design method for state space systems with stochastic parameters are needed. or this problem, De Koning proposed a controller design method for state space systems with stochastic system parameters De Koning 1982 which employs standard quadratic cost function for optimal control. Although De Koning s paper can take care of the effect of the variation of system parameters, it cannot suppress the variance of the transient of the resulting control system caused by the variation of the system parameters. The present paper proposes a novel optimal control method with variance suppression for state space systems with stochastic parameters and disturbances, in which we can suppress the variance of the trajectory of the state caused by the variation of the system parameters. urthermore, numerical simulations demonstrate the effectiveness of the proposed method. We believe that the proposed method can provide a new framework to stochastic control together with Bayesian estimation. 2. PRELIMINARIES This section gives notations and some preliminary results according to Bishop 26; De Koning The symbol R n denotes the n-dimensional Euclidean space. M mn and M n denote the space of m n real valued rectangular matrices and that of n n real valued square matrices, respectively. S n denotes the space of n n real valued symmetric matrices. The symbol vec denotes a function satisfying a 1 a 2 a 1i veca =. a 2i Rmn, a i :=. Rm. a n a mi
2 where a ij is the i, j element of the matrix A. The expectation of a time-varying stochastic parameter a t is denoted without the time parameter t as E[a] instead of E[a t ], if its statistics is time invariant. We use several transformations for symmetric matrices. Let us define a monotonic transformation. Definition 1. De Koning 1982 A transformation for symmetric matrices A : S n S n is said to be monotonic if it satisfies AX AY, for any symmetric matrices X, Y S n satisfying X Y. The following lemma holds for monotonic transformations. Lemma 1. De Koning 1982 Suppose that a transformation A : S n S n is monotonic and that AX holds for any symmetric matrix X. Then A i is monotonic for all natural number i and A i X holds for any symmetric matrix X. Next, let us consider a discrete time system x t+1 = A t x t + B t u t 1 where x t R n is the state, u t R m is the control input, A t M n, B t M nm are the system matrices costing of system parameters. Suppose that the system matrices A t and B t are random parameters with time invariant statistics. Assume also that the initial state x is deterministic. Apply a state feedback u t = Lx t 2 to the system 1 with the feedback gain L M mn. Then the resulting feedback system is described by x t+1 = Ψ L,t x t 3 with a new system matrix Ψ L,t := A t B t L M n. In order to describe the behavior of the state x t with its variance, let us define a transformation A L : S n S n describing the expectation of a quadratic function along the feedback system 3 as follows. A L X := E[Ψ T LXΨ L ], X S n 4 The transformation A L thus defined satisfies the following lemma. Lemma 2. De Koning 1982 a E[x T i Xx i x ] = x T A i L Xx holds for any natural number i and any symmetric matrix X. b The transformation A i L is linear and monotonic for any natural number i. 3. OPTIMAL CONTROL OR VARIANCE SUPPRESSION This section states the main result of the paper, optimal control for variance suppression. Consider a linear discretetime system with stochastic parameters x t+1 = A t x t + B t u t + G t ɛ t 5 where ɛ t R n is a stochastic external noise. A t M n, B t M nm, and G t M n are stochastic parameters defined by time invariant statistics. or this system, let us consider the following cost function. J N U N, x = E x T t Qx t + u T t Ru t + trs cov[x t+1 x t ] } + x TN x N x ] Here the matrices Q S n, R S m, S n, and S S n are design parameters and U N = u t, t N 1} denotes the collection of the input. The third term in the right hand side of Equation 6 reduces to tr S cov[x t+1 x t ] = E [ x T t+1sx t+1 x t ] E[xt+1 x t ] T S E[x t+1 x t ] which is the weighted sum of the covariance of the states. The coefficient matrix S can be used to select the weights between the variation and the average of the state. or this system, the optimal control problem for variance suppression is defined as follows. Definition 2. Consider the system 5 and the cost function 6. or a given initial condition x R n, find an input sequence UN = u t, t N 1} minimizing the cost function J N U N, x and the corresponding minimum value of the cost function JN x. We call this control problem as a finite time optimal control for variance suppression. Before solving the variance suppression problem, let us consider the relationship between the value of the cost function at the time t with the state x t and that at the time t 1 with the state x t 1. Suppose that a state feedback u t = Lx t is employed, then the term to evaluate the variance of the state E[trS cov[x t+1 x t ] x ] in the cost function can be calculated as 6 E [trs cov[x t+1 x t ] x ] = E [ E[x T t+1sx t+1 x t ] E[x t+1 x t ] T ] S E[x t+1 x t ] x = E [ x T t E[Ψ T L SΨ L ] E[Ψ T L]S E [Ψ L ] ] x t x = E [ x T t AL S E[Ψ T L]S E[Ψ L ] ] x t x = E [ x T t 1A L AL S E[Ψ T L]S E[Ψ L ] ] x t 1 x = E [ x T A t L S A t 1 L E[Ψ T L ]S E[Ψ L ] } ] x x = x T A t L S A t 1 L E[Ψ T L ]S E[Ψ L ] } x 7 where the function B L : S n S n is defined by B L X := A L X+Q+L T RL+A L S E[Ψ T L]S E[Ψ L ], with X S n 8 βx := E[ɛ T G T XGɛ]. 9 Then the value of the cost function 6 becomes
3 J N U N, x = E x T t Q + L T RLx t } ] + trs cov[x t+1 x t ] + x TN x N x N 1 Q + L T RL = x T A t L } + A L S E[Ψ T L]S E[Ψ L ] + A N L + β N 1 = x T B N L x + β t 1 A t L + A j L x Q + L T RL + A L S E[Ψ T L]S E[Ψ L ] } + NS NS + N 1 B t L. 1 Now we can prove a property of the function B L as in the following lemma. Lemma 3. or any natural number i, the function B i L is monotonic. In particular, for any symmetric matrix X, B i L X holds. Proof. irst of all, it will be proved that B i L = B L is monotonic for the case i = 1. Lemma 2 b implies that A L X A L Y holds for any matrices X, Y S n satisfying X Y since the function A L is linear. Hence Equation 8 proves B L X B L Y which implies the monotonicity of B L. On the other hand, A L X + Q + L T RL holds for any X. Therefore what we have to prove is the property A L S E[Ψ L ] T S E[Ψ L ] which will complete the proof by using Lemma 1. Since S S n, there exists an orthogonal matrix M M n diagonalizing S as Σ = M T SM = diags 1, s 2,..., s n. Employing a coordinate transformation y t = M T x t, the closed loop system 3 is described by y t+1 = M T Ψ L,t My t. Using the expression y t = [yt 1, yt 2,, yt n ] T, we have AL S E[Ψ T L]S E[Ψ L ] x t x T t = y T t M T A L S E[Ψ T L]S E[Ψ L ] My t = yt T M T E[Ψ T LSΨ L ] E[Ψ T L]S E[Ψ L ] My t = yt T E[M T Ψ T LMΣM T Ψ L M] E[M T Ψ T LM]Σ E[M T Ψ L M] = E[yt+1Σy T t+1 y t ] E[y t+1 y t ] T Σ E[y t+1 y t ] = tr Σ cov[y t+1 y t ] n = s i var[yt+1 y i t ]. i=1 y t Since the variances var[yt+1 y i t ] s and the coefficients s i s are positive, n i=1 s i var[yt+1 y i t ] which proves A L S E[Ψ T L ]S E[Ψ L]. This proves that B L X = A L X + Q + L T RL + A L S E[Ψ T L ]S E[Ψ L] holds for any X. It follows from Lemma 1 that the function BL i is monotonic for any natural number i and that BL i X holds for any X which completes the proof. Using this lemma, we can prove the main theorem which provides the solution to the optimal control for variance suppression. Theorem 1. The optimal control law to minimize the cost function 6 and the minimum value of the cost function J N x are given as follows. u t = L B N t 1 x t, t =,..., N 1 11 J N x = x T B N x + β NS + N 1 B t Here the function B : S n S n is defined by, x 12 B X := B LX X, X S n. 13 The function B LX is defined in Equation 8 and the gain matrix L X is defined by L X := E[B T XB]+Σ BB +R 1 E[B T XA]+Σ BA, where Σ XY := E[X T SY ] E[X] T S E[Y ]. Proof. irst of all, let us define the value function by V t x t = min u t,,u N 1 E j=t x T j Qx j + u T j Ru j X S n 14 } + trs cov[x j+1 x j ] + x T N x N x t ]. 15 which denotes the minimum value of the cost function along the system 5 for the time period from t to N 1 with the initial state x t. Let us now prove by induction that k 1 V N k x N k = x T N kb k x N k + β ks + B t holds for any natural number k1 k N. 16 The case k = : The boundary condition implies that V N x N = x T N x N = x T N B x N. The case k = 1:
4 V N 1 x N 1 [ u N 1 u N 1 E x T N 1Qx N 1 + u T N 1Ru N 1 + cov[x N x N 1 ] + x T N x N x N 1 ] x T N 1 E[A T B A] + Σ AA + Q + u T N 1 E[B T B B] + Σ BB + R + 2u T N 1 E[B T B A] + Σ BA } + E[ɛ T G T B Gɛ] + E[ɛ T G T SGɛ] x N 1 x N 1 u N holds. Hence the optimal input u N 1 minimizing the right hand side of Equation 17 is given by u N 1 = E[B T B B] + Σ BB + R 1 E[B T B A] + Σ BA x N 1 = L B x N V N l x N l u N l,,u N 1 u N l u N l E j=n l x T j Qx j + u T j Ru j + tr S cov[x j+1 x j ] } + x T N x N x N l ] x T N lqx N l + u T N lru N l + tr S cov[x N l+1 x N l ] + E[V N l+1 x N l+1 x N l ] x T N l + u T N l + 2u T N l E[A T B A] + Σ AA + Q x N l E[B T B B] + Σ BB + R u N l E[B T B A] + Σ BA x N l + E[ɛ T G T B Gɛ] + E[ɛ T G T SGɛ] } l 2 + β l 1S + B j. 2 Substituting this equation for Equation 17, we obtain V N 1 x N 1 = x T N 1 E[A T B A] + Σ AA + Q x N 1 + x T N 1L T B E[B T B B] + Σ BB + R L B x N 1 2x T N 1L T B E[B T B A] + Σ BA x N 1 + β S + B = x T N 1 E [ A BL B T B A BL B ] + Q + L T B RL B + E [ A BL B T SA BL B ] E [ A BL B ] T [ S E A BLB ]} x N 1 + β S + B = x T N 1 A LB B + Q + L T B RL B + A S E[Ψ LB L B ]T S E[Ψ LB x ] N 1 + β S + B = x T N 1B LB B x N 1 + β S + B = x T N 1B 1 x N 1 + β S + B. 19 The input u N l minimizing the right hand side of Equation 2 is u N l = E[B T B B] + Σ BB + R 1 E[B T B A] + Σ BA x N l = L B x N l. 21 Substituting this input for Equation 2, we obtain V N l x N l = x T N l E [ A BL B T B A BL B ] + Q + L T RL B B + E [ A BL B T SA BL B ] E [ A BL B ] T [ S E A BLB ]} x N l + β ls + B j = x T N l = x T N lb LB B x N l + β ls + B j Therefore, Equation 16 holds for the case k = 1. Next, suppose that Equation 16 holds in the case k = l1 l N 1 and prove 16 in the case k = l + 1: = x T N lb l x N l + β ls + B j 22 Namely, we suppose V N l+1 x N l+1 = x T N l+1 B x N l+1 + β l 1S + l 2 Bj here. Then we have which coincides with Equation 16 in the case k = l + 1. A LB B + A LB S E[Ψ LB + β ls + B j + Q + L T B RL B ] T S E[Ψ LB ] x N l
5 This implies that Equation 16 holds for any k 1 k N by induction. Substituting k = N t for Equation 16, we obtain the optimal input u t at the time t as u t = E[B T B N t 1 B] + Σ BB + R 1 E[B T B N t 1 A] + Σ BA x t = L B N t+1 x t. Next, substituting k = N for Equation 16, we have the minimum value of the cost function as J N x = V x = x T B N x + β NS + N 1 B t Conversely, if we employ the input in Equation 11, then the cost function 6 coincides with its minimum 12. This prove the theorem. Let us denote the value of the cost function at the time t by V t x t = x T Π t x. Then Equation 13 reduces to the following recursive equation similar to a Riccati equation. Π t 1 = Q + Σ AA + E[A T Π t A] E[A T Π t B] + Σ AB E[B T Π t B] + Σ BB + R 1 E[B T Π t A] + Σ BA. 23 Here Π N = and B k = Π N k, namely, u t = L Πt x t. urthermore, Equation 23 reduces to a Riccati equation for the conventional LQG problem if the parameters A t and B t are deterministic, which implies that the proposed method is a natural generalization of the conventional LQG method. 4. NUMERICAL EXAMPLE This section gives a numerical example to demonstrate the effectiveness of the proposed method in comparison to the conventional LQG method. Let us consider the plant described by Equation 5 with the terminal time N = 25, the state x R 2 and the input u R. [ ] [ ] 1.1 E[A] =, E[B] = cov [vec[a, B]]= [ ] [ ] 1 4 Q =, R = 1, = 1 4 [ ] 1 x = 1 The covariance cov [vec[a, B]] are selected in such a way that the parameters A 21, A 22 and B 2 have 1% standard deviations. ig.1 shows the time responses of 1 random samples of the feedback system with a conventional LQG controller. igs.2 and 3 show the time responses of 1 random samples by the proposed method with the design parameters S = and S = 1I, respectively. In those figures, the plus signs + denote the upper and the lower i State transition of x 1 ig. 1. State transition LQG ii State transition of x 2 line of the 1σ deviation from the average and the solid lines denote the sampled responses. igures show that both the conventional LQG controller and the proposed controller achieve stability of both the average and the variance. The parameter S = 1I in ig.3 achieves small variance as we expected whereas the convergence speed of its average is slower, which implies that there is a trade-off between the convergence speed of the expectation the that of the variance. i State transition of x 1 ii State transition of x 2 ig. 2. State transition proposed method S = i State transition of x 1 ii State transition of x 2 ig. 3. State transition proposed method S = 1I Next, igs.4 6 show the case in which the variances of the parameters are bigger than the case in igs.1 3. The covariance of the system parameters A and B is selected as follows in such a way that the parameters A 21, A 22 and B 2 have 5% standard deviations cov [vec[a, B]] = igs.4 6 show the time responses of 1 random samples by the LQG method and the proposed method with the design parameters S = and S = 1I, respectively. In those figures, the plus signs + denote the upper and the
6 ACKNOWLEDGEMENTS i State transition of x 1 ig. 4. State transition LQG i State transition of x 1 ii State transition of x 2 ii State transition of x 2 ig. 5. State transition proposed method S = i State transition of x 1 ii State transition of x 2 ig. 6. State transition proposed method S = 1I lower line of the 1σ deviation from the average and the solid lines denote the sampled responses as in igs.1 6. ig.4 shows the time responses of the state of the feedback system with the LQG controller. In the figure the states diverge, since the LQG controller does not take care of the variance of the system parameters. igs.5 and 6 shows the time responses of the states with the proposed controllers for the parameters S = and S = 1I. Both figures show that the state converge to smoothly and the variance of the state is smaller in the case S = 1I compared to the other case S =. This result shows that the controller with the bigger parameter S suppresses the variance of the state response as we desired, whereas it also let the convergence speed of the average of the state response slower as in the previous case. The authors would like to thank Professor Yasumasa ujisaki at Osaka University for his valuable suggestions. REERENCES Aström, K.J. 26. Introduction to Stochastic Control Theory. Dover Publications. Barber, D. and Chiappa, S. 27. Unified inference for variational Bayesian linear Gaussian state-space models. In Advances in Neural Information Processing Systems 19 NIPS 2, The MIT Press. Bishop, C.M. 26. Pattern Recognition and Machine Learning. Springer, New York. Calafiore, G., Tempo, R., and Dabbene,. 24. Randomized Algorithms for Analysis and Control of Uncertain Systems. Springer. De Koning, W.L Infinite horizon optimal control of linear discrete time systems with stochastic parameters. Automatica, 184, ukunaga, S., Ishihara, Y., and ujimoto, K. 28. Byesian estimation methods for state-space models. In Proc. SICE 8th Annual Conference on Control Systems. in Japanese. Kano, M. 21. Modeling from process data. Measurement and Control, 492, in Japanese. Mendel, J.M. and Gieseking, D.L Bibliography on the Linear-Quadratic-Gaussian problem. IEEE Trans. Autom. Contr., 6, Sain, M.K Control of linear systems according to the minimal variance criterion: A new approach to the disturbance problem. IEEE Trans. Autom. Contr., 111, Sain, M.K., Won, C.H., and Spencer Jr, B Cumulants in risk-sensitive contro: The full-state-feedback cost variance case risk-sensitive and MCV stochastic control. In Proc. 34th IEEE Conf. on Decision and Control, Whittle, P Risk-sensitive Linear/Quadratic/Gaussian control. Advances in Applied Probability, 13, CONCLUSION This paper proposes a novel optimal control method with variance suppression for state space systems with stochastic parameters and disturbances. We have derived a Riccati type recursive equation to solve the proposed optimal control problem which employs second order statistics of the system parameters. urther, some numerical simulations demonstrate the effectiveness of the proposed method. We believe that the proposed method provides a new stochastic control framework with Bayesian estimation.
Optimal Control of Linear Systems with Stochastic Parameters for Variance Suppression
Optimal Control of inear Systems with Stochastic Parameters for Variance Suppression Kenji Fujimoto, Yuhei Ota and Makishi Nakayama Abstract In this paper, we consider an optimal control problem for a
More informationLinear-Quadratic-Gaussian (LQG) Controllers and Kalman Filters
Linear-Quadratic-Gaussian (LQG) Controllers and Kalman Filters Emo Todorov Applied Mathematics and Computer Science & Engineering University of Washington Winter 204 Emo Todorov (UW) AMATH/CSE 579, Winter
More informationThe Uncertainty Threshold Principle: Some Fundamental Limitations of Optimal Decision Making under Dynamic Uncertainty
The Uncertainty Threshold Principle: Some Fundamental Limitations of Optimal Decision Making under Dynamic Uncertainty Michael Athans, Richard Ku, Stanley Gershwin (Nicholas Ballard 538477) Introduction
More informationPerformance assessment of MIMO systems under partial information
Performance assessment of MIMO systems under partial information H Xia P Majecki A Ordys M Grimble Abstract Minimum variance (MV) can characterize the most fundamental performance limitation of a system,
More informationIn: Proc. BENELEARN-98, 8th Belgian-Dutch Conference on Machine Learning, pp 9-46, 998 Linear Quadratic Regulation using Reinforcement Learning Stephan ten Hagen? and Ben Krose Department of Mathematics,
More informationLecture 5 Linear Quadratic Stochastic Control
EE363 Winter 2008-09 Lecture 5 Linear Quadratic Stochastic Control linear-quadratic stochastic control problem solution via dynamic programming 5 1 Linear stochastic system linear dynamical system, over
More informationStatistical Control for Performance Shaping using Cost Cumulants
This is the author's version of an article that has been published in this journal Changes were made to this version by the publisher prior to publication The final version of record is available at http://dxdoiorg/1119/tac1788
More informationFIR Filters for Stationary State Space Signal Models
Proceedings of the 17th World Congress The International Federation of Automatic Control FIR Filters for Stationary State Space Signal Models Jung Hun Park Wook Hyun Kwon School of Electrical Engineering
More informationAdaptive Filters. un [ ] yn [ ] w. yn n wun k. - Adaptive filter (FIR): yn n n w nun k. (1) Identification. Unknown System + (2) Inverse modeling
Adaptive Filters - Statistical digital signal processing: in many problems of interest, the signals exhibit some inherent variability plus additive noise we use probabilistic laws to model the statistical
More informationEL2520 Control Theory and Practice
EL2520 Control Theory and Practice Lecture 8: Linear quadratic control Mikael Johansson School of Electrical Engineering KTH, Stockholm, Sweden Linear quadratic control Allows to compute the controller
More informationLecture 10 Linear Quadratic Stochastic Control with Partial State Observation
EE363 Winter 2008-09 Lecture 10 Linear Quadratic Stochastic Control with Partial State Observation partially observed linear-quadratic stochastic control problem estimation-control separation principle
More informationA Matrix Theoretic Derivation of the Kalman Filter
A Matrix Theoretic Derivation of the Kalman Filter 4 September 2008 Abstract This paper presents a matrix-theoretic derivation of the Kalman filter that is accessible to students with a strong grounding
More informationStochastic Tube MPC with State Estimation
Proceedings of the 19th International Symposium on Mathematical Theory of Networks and Systems MTNS 2010 5 9 July, 2010 Budapest, Hungary Stochastic Tube MPC with State Estimation Mark Cannon, Qifeng Cheng,
More informationA NOVEL OPTIMAL PROBABILITY DENSITY FUNCTION TRACKING FILTER DESIGN 1
A NOVEL OPTIMAL PROBABILITY DENSITY FUNCTION TRACKING FILTER DESIGN 1 Jinglin Zhou Hong Wang, Donghua Zhou Department of Automation, Tsinghua University, Beijing 100084, P. R. China Control Systems Centre,
More informationVisual feedback Control based on image information for the Swirling-flow Melting Furnace
Visual feedback Control based on image information for the Swirling-flow Melting Furnace Paper Tomoyuki Maeda Makishi Nakayama Hirokazu Araya Hiroshi Miyamoto Member In this paper, a new visual feedback
More informationLinear Dimensionality Reduction
Outline Hong Chang Institute of Computing Technology, Chinese Academy of Sciences Machine Learning Methods (Fall 2012) Outline Outline I 1 Introduction 2 Principal Component Analysis 3 Factor Analysis
More information3. ARMA Modeling. Now: Important class of stationary processes
3. ARMA Modeling Now: Important class of stationary processes Definition 3.1: (ARMA(p, q) process) Let {ɛ t } t Z WN(0, σ 2 ) be a white noise process. The process {X t } t Z is called AutoRegressive-Moving-Average
More informationAPPROXIMATE SOLUTION OF A SYSTEM OF LINEAR EQUATIONS WITH RANDOM PERTURBATIONS
APPROXIMATE SOLUTION OF A SYSTEM OF LINEAR EQUATIONS WITH RANDOM PERTURBATIONS P. Date paresh.date@brunel.ac.uk Center for Analysis of Risk and Optimisation Modelling Applications, Department of Mathematical
More informationRiccati difference equations to non linear extended Kalman filter constraints
International Journal of Scientific & Engineering Research Volume 3, Issue 12, December-2012 1 Riccati difference equations to non linear extended Kalman filter constraints Abstract Elizabeth.S 1 & Jothilakshmi.R
More informationA Tour of Reinforcement Learning The View from Continuous Control. Benjamin Recht University of California, Berkeley
A Tour of Reinforcement Learning The View from Continuous Control Benjamin Recht University of California, Berkeley trustable, scalable, predictable Control Theory! Reinforcement Learning is the study
More informationDISCRETE-TIME ADAPTIVE LQG/LTR CONTROL
DISCRETE-TIME ADAPTIVE LQG/LTR CONTROL DARIUSZ HORLA Poznan University of Technology Inst. of Control and Inf. Eng. ul. Piotrowo 3a, 6-965 Poznan POLAND ANDRZEJ KRÓLIKOWSKI Poznan University of Technology
More informationChapter 3. LQ, LQG and Control System Design. Dutch Institute of Systems and Control
Chapter 3 LQ, LQG and Control System H 2 Design Overview LQ optimization state feedback LQG optimization output feedback H 2 optimization non-stochastic version of LQG Application to feedback system design
More informationMachine Learning for Large-Scale Data Analysis and Decision Making A. Week #1
Machine Learning for Large-Scale Data Analysis and Decision Making 80-629-17A Week #1 Today Introduction to machine learning The course (syllabus) Math review (probability + linear algebra) The future
More informationOptimal Polynomial Control for Discrete-Time Systems
1 Optimal Polynomial Control for Discrete-Time Systems Prof Guy Beale Electrical and Computer Engineering Department George Mason University Fairfax, Virginia Correspondence concerning this paper should
More informationRisk-Sensitive Filtering and Smoothing via Reference Probability Methods
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 42, NO. 11, NOVEMBER 1997 1587 [5] M. A. Dahleh and J. B. Pearson, Minimization of a regulated response to a fixed input, IEEE Trans. Automat. Contr., vol.
More informationNoise Reduction of JPEG Images by Sampled-Data H Optimal ε Filters
SICE Annual Conference 25 in Okayama, August 8-1, 25 Okayama University, Japan Noise Reduction of JPEG Images by Sampled-Data H Optimal ε Filters H. Kakemizu,1, M. Nagahara,2, A. Kobayashi,3, Y. Yamamoto,4
More informationMA 575 Linear Models: Cedric E. Ginestet, Boston University Revision: Probability and Linear Algebra Week 1, Lecture 2
MA 575 Linear Models: Cedric E Ginestet, Boston University Revision: Probability and Linear Algebra Week 1, Lecture 2 1 Revision: Probability Theory 11 Random Variables A real-valued random variable is
More informationManifold Learning for Signal and Visual Processing Lecture 9: Probabilistic PCA (PPCA), Factor Analysis, Mixtures of PPCA
Manifold Learning for Signal and Visual Processing Lecture 9: Probabilistic PCA (PPCA), Factor Analysis, Mixtures of PPCA Radu Horaud INRIA Grenoble Rhone-Alpes, France Radu.Horaud@inria.fr http://perception.inrialpes.fr/
More information4 Derivations of the Discrete-Time Kalman Filter
Technion Israel Institute of Technology, Department of Electrical Engineering Estimation and Identification in Dynamical Systems (048825) Lecture Notes, Fall 2009, Prof N Shimkin 4 Derivations of the Discrete-Time
More informationData Analysis and Manifold Learning Lecture 6: Probabilistic PCA and Factor Analysis
Data Analysis and Manifold Learning Lecture 6: Probabilistic PCA and Factor Analysis Radu Horaud INRIA Grenoble Rhone-Alpes, France Radu.Horaud@inrialpes.fr http://perception.inrialpes.fr/ Outline of Lecture
More informationOptimal control and estimation
Automatic Control 2 Optimal control and estimation Prof. Alberto Bemporad University of Trento Academic year 2010-2011 Prof. Alberto Bemporad (University of Trento) Automatic Control 2 Academic year 2010-2011
More informationOn Stochastic Adaptive Control & its Applications. Bozenna Pasik-Duncan University of Kansas, USA
On Stochastic Adaptive Control & its Applications Bozenna Pasik-Duncan University of Kansas, USA ASEAS Workshop, AFOSR, 23-24 March, 2009 1. Motivation: Work in the 1970's 2. Adaptive Control of Continuous
More information[POLS 8500] Review of Linear Algebra, Probability and Information Theory
[POLS 8500] Review of Linear Algebra, Probability and Information Theory Professor Jason Anastasopoulos ljanastas@uga.edu January 12, 2017 For today... Basic linear algebra. Basic probability. Programming
More informationLinear State Feedback Controller Design
Assignment For EE5101 - Linear Systems Sem I AY2010/2011 Linear State Feedback Controller Design Phang Swee King A0033585A Email: king@nus.edu.sg NGS/ECE Dept. Faculty of Engineering National University
More informationOnline monitoring of MPC disturbance models using closed-loop data
Online monitoring of MPC disturbance models using closed-loop data Brian J. Odelson and James B. Rawlings Department of Chemical Engineering University of Wisconsin-Madison Online Optimization Based Identification
More informationCheng Soon Ong & Christian Walder. Canberra February June 2018
Cheng Soon Ong & Christian Walder Research Group and College of Engineering and Computer Science Canberra February June 2018 (Many figures from C. M. Bishop, "Pattern Recognition and ") 1of 89 Part II
More informationLecture 9: Discrete-Time Linear Quadratic Regulator Finite-Horizon Case
Lecture 9: Discrete-Time Linear Quadratic Regulator Finite-Horizon Case Dr. Burak Demirel Faculty of Electrical Engineering and Information Technology, University of Paderborn December 15, 2015 2 Previous
More informationQuadratic Extended Filtering in Nonlinear Systems with Uncertain Observations
Applied Mathematical Sciences, Vol. 8, 2014, no. 4, 157-172 HIKARI Ltd, www.m-hiari.com http://dx.doi.org/10.12988/ams.2014.311636 Quadratic Extended Filtering in Nonlinear Systems with Uncertain Observations
More informationStatistical Control for Performance Shaping using Cost Cumulants
SUBMITTED TO IEEE TRANSACTION ON AUTOMATIC CONTROL 1 Statistical Control for Performance Shaping using Cost Cumulants Bei Kang Member IEEE Chukwuemeka Aduba Member IEEE and Chang-Hee Won Member IEEE Abstract
More informationMultirate MVC Design and Control Performance Assessment: a Data Driven Subspace Approach*
Multirate MVC Design and Control Performance Assessment: a Data Driven Subspace Approach* Xiaorui Wang Department of Electrical and Computer Engineering University of Alberta Edmonton, AB, Canada T6G 2V4
More informationChapter 2 Event-Triggered Sampling
Chapter Event-Triggered Sampling In this chapter, some general ideas and basic results on event-triggered sampling are introduced. The process considered is described by a first-order stochastic differential
More informationCross Directional Control
Cross Directional Control Graham C. Goodwin Day 4: Lecture 4 16th September 2004 International Summer School Grenoble, France 1. Introduction In this lecture we describe a practical application of receding
More informationEEL 851: Biometrics. An Overview of Statistical Pattern Recognition EEL 851 1
EEL 851: Biometrics An Overview of Statistical Pattern Recognition EEL 851 1 Outline Introduction Pattern Feature Noise Example Problem Analysis Segmentation Feature Extraction Classification Design Cycle
More informationStatistics 910, #15 1. Kalman Filter
Statistics 910, #15 1 Overview 1. Summary of Kalman filter 2. Derivations 3. ARMA likelihoods 4. Recursions for the variance Kalman Filter Summary of Kalman filter Simplifications To make the derivations
More informationFactor Analysis (10/2/13)
STA561: Probabilistic machine learning Factor Analysis (10/2/13) Lecturer: Barbara Engelhardt Scribes: Li Zhu, Fan Li, Ni Guan Factor Analysis Factor analysis is related to the mixture models we have studied.
More informationLecture 4: Least Squares (LS) Estimation
ME 233, UC Berkeley, Spring 2014 Xu Chen Lecture 4: Least Squares (LS) Estimation Background and general solution Solution in the Gaussian case Properties Example Big picture general least squares estimation:
More informationChapter 5 Matrix Approach to Simple Linear Regression
STAT 525 SPRING 2018 Chapter 5 Matrix Approach to Simple Linear Regression Professor Min Zhang Matrix Collection of elements arranged in rows and columns Elements will be numbers or symbols For example:
More information2. Matrix Algebra and Random Vectors
2. Matrix Algebra and Random Vectors 2.1 Introduction Multivariate data can be conveniently display as array of numbers. In general, a rectangular array of numbers with, for instance, n rows and p columns
More informationStochastic and Adaptive Optimal Control
Stochastic and Adaptive Optimal Control Robert Stengel Optimal Control and Estimation, MAE 546 Princeton University, 2018! Nonlinear systems with random inputs and perfect measurements! Stochastic neighboring-optimal
More informationTilburg University. An equivalence result in linear-quadratic theory van den Broek, W.A.; Engwerda, Jacob; Schumacher, Hans. Published in: Automatica
Tilburg University An equivalence result in linear-quadratic theory van den Broek, W.A.; Engwerda, Jacob; Schumacher, Hans Published in: Automatica Publication date: 2003 Link to publication Citation for
More informationRECURSIVE ESTIMATION AND KALMAN FILTERING
Chapter 3 RECURSIVE ESTIMATION AND KALMAN FILTERING 3. The Discrete Time Kalman Filter Consider the following estimation problem. Given the stochastic system with x k+ = Ax k + Gw k (3.) y k = Cx k + Hv
More informationSliding Window Recursive Quadratic Optimization with Variable Regularization
11 American Control Conference on O'Farrell Street, San Francisco, CA, USA June 29 - July 1, 11 Sliding Window Recursive Quadratic Optimization with Variable Regularization Jesse B. Hoagg, Asad A. Ali,
More informationCS229 Lecture notes. Andrew Ng
CS229 Lecture notes Andrew Ng Part X Factor analysis When we have data x (i) R n that comes from a mixture of several Gaussians, the EM algorithm can be applied to fit a mixture model. In this setting,
More informationCONSTRAINED MODEL PREDICTIVE CONTROL ON CONVEX POLYHEDRON STOCHASTIC LINEAR PARAMETER VARYING SYSTEMS. Received October 2012; revised February 2013
International Journal of Innovative Computing, Information and Control ICIC International c 2013 ISSN 1349-4198 Volume 9, Number 10, October 2013 pp 4193 4204 CONSTRAINED MODEL PREDICTIVE CONTROL ON CONVEX
More information2 Introduction of Discrete-Time Systems
2 Introduction of Discrete-Time Systems This chapter concerns an important subclass of discrete-time systems, which are the linear and time-invariant systems excited by Gaussian distributed stochastic
More informationState-space Model. Eduardo Rossi University of Pavia. November Rossi State-space Model Financial Econometrics / 49
State-space Model Eduardo Rossi University of Pavia November 2013 Rossi State-space Model Financial Econometrics - 2013 1 / 49 Outline 1 Introduction 2 The Kalman filter 3 Forecast errors 4 State smoothing
More informationKalman Filter and Parameter Identification. Florian Herzog
Kalman Filter and Parameter Identification Florian Herzog 2013 Continuous-time Kalman Filter In this chapter, we shall use stochastic processes with independent increments w 1 (.) and w 2 (.) at the input
More information3. Probability and Statistics
FE661 - Statistical Methods for Financial Engineering 3. Probability and Statistics Jitkomut Songsiri definitions, probability measures conditional expectations correlation and covariance some important
More informationSYNTHESIS OF ROBUST DISCRETE-TIME SYSTEMS BASED ON COMPARISON WITH STOCHASTIC MODEL 1. P. V. Pakshin, S. G. Soloviev
SYNTHESIS OF ROBUST DISCRETE-TIME SYSTEMS BASED ON COMPARISON WITH STOCHASTIC MODEL 1 P. V. Pakshin, S. G. Soloviev Nizhny Novgorod State Technical University at Arzamas, 19, Kalinina ul., Arzamas, 607227,
More informationOn Expected Gaussian Random Determinants
On Expected Gaussian Random Determinants Moo K. Chung 1 Department of Statistics University of Wisconsin-Madison 1210 West Dayton St. Madison, WI 53706 Abstract The expectation of random determinants whose
More information4F3 - Predictive Control
4F3 Predictive Control - Lecture 2 p 1/23 4F3 - Predictive Control Lecture 2 - Unconstrained Predictive Control Jan Maciejowski jmm@engcamacuk 4F3 Predictive Control - Lecture 2 p 2/23 References Predictive
More informationEE C128 / ME C134 Feedback Control Systems
EE C128 / ME C134 Feedback Control Systems Lecture Additional Material Introduction to Model Predictive Control Maximilian Balandat Department of Electrical Engineering & Computer Science University of
More informationStability of Hybrid Control Systems Based on Time-State Control Forms
Stability of Hybrid Control Systems Based on Time-State Control Forms Yoshikatsu HOSHI, Mitsuji SAMPEI, Shigeki NAKAURA Department of Mechanical and Control Engineering Tokyo Institute of Technology 2
More informationIntroduction p. 1 Fundamental Problems p. 2 Core of Fundamental Theory and General Mathematical Ideas p. 3 Classical Statistical Decision p.
Preface p. xiii Acknowledgment p. xix Introduction p. 1 Fundamental Problems p. 2 Core of Fundamental Theory and General Mathematical Ideas p. 3 Classical Statistical Decision p. 4 Bayes Decision p. 5
More informationLecture Notes 1: Vector spaces
Optimization-based data analysis Fall 2017 Lecture Notes 1: Vector spaces In this chapter we review certain basic concepts of linear algebra, highlighting their application to signal processing. 1 Vector
More informationComputer Vision Group Prof. Daniel Cremers. 9. Gaussian Processes - Regression
Group Prof. Daniel Cremers 9. Gaussian Processes - Regression Repetition: Regularized Regression Before, we solved for w using the pseudoinverse. But: we can kernelize this problem as well! First step:
More informationCS281 Section 4: Factor Analysis and PCA
CS81 Section 4: Factor Analysis and PCA Scott Linderman At this point we have seen a variety of machine learning models, with a particular emphasis on models for supervised learning. In particular, we
More informationTAMS39 Lecture 2 Multivariate normal distribution
TAMS39 Lecture 2 Multivariate normal distribution Martin Singull Department of Mathematics Mathematical Statistics Linköping University, Sweden Content Lecture Random vectors Multivariate normal distribution
More informationADAPTIVE FILTER THEORY
ADAPTIVE FILTER THEORY Fifth Edition Simon Haykin Communications Research Laboratory McMaster University Hamilton, Ontario, Canada International Edition contributions by Telagarapu Prabhakar Department
More informationGaussian processes. Chuong B. Do (updated by Honglak Lee) November 22, 2008
Gaussian processes Chuong B Do (updated by Honglak Lee) November 22, 2008 Many of the classical machine learning algorithms that we talked about during the first half of this course fit the following pattern:
More informationSCUOLA DI SPECIALIZZAZIONE IN FISICA MEDICA. Sistemi di Elaborazione dell Informazione. Regressione. Ruggero Donida Labati
SCUOLA DI SPECIALIZZAZIONE IN FISICA MEDICA Sistemi di Elaborazione dell Informazione Regressione Ruggero Donida Labati Dipartimento di Informatica via Bramante 65, 26013 Crema (CR), Italy http://homes.di.unimi.it/donida
More informationA L A BA M A L A W R E V IE W
A L A BA M A L A W R E V IE W Volume 52 Fall 2000 Number 1 B E F O R E D I S A B I L I T Y C I V I L R I G HT S : C I V I L W A R P E N S I O N S A N D TH E P O L I T I C S O F D I S A B I L I T Y I N
More informationOptimal Decentralized Control of Coupled Subsystems With Control Sharing
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 58, NO. 9, SEPTEMBER 2013 2377 Optimal Decentralized Control of Coupled Subsystems With Control Sharing Aditya Mahajan, Member, IEEE Abstract Subsystems that
More information6.241 Dynamic Systems and Control
6.241 Dynamic Systems and Control Lecture 24: H2 Synthesis Emilio Frazzoli Aeronautics and Astronautics Massachusetts Institute of Technology May 4, 2011 E. Frazzoli (MIT) Lecture 24: H 2 Synthesis May
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 4
EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 4 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory April 12, 2012 Andre Tkacenko
More informationComputer Vision Group Prof. Daniel Cremers. 4. Gaussian Processes - Regression
Group Prof. Daniel Cremers 4. Gaussian Processes - Regression Definition (Rep.) Definition: A Gaussian process is a collection of random variables, any finite number of which have a joint Gaussian distribution.
More informationVariational Principal Components
Variational Principal Components Christopher M. Bishop Microsoft Research 7 J. J. Thomson Avenue, Cambridge, CB3 0FB, U.K. cmbishop@microsoft.com http://research.microsoft.com/ cmbishop In Proceedings
More informationNumerical atmospheric turbulence models and LQG control for adaptive optics system
Numerical atmospheric turbulence models and LQG control for adaptive optics system Jean-Pierre FOLCHER, Marcel CARBILLET UMR6525 H. Fizeau, Université de Nice Sophia-Antipolis/CNRS/Observatoire de la Côte
More informationState-space Model. Eduardo Rossi University of Pavia. November Rossi State-space Model Fin. Econometrics / 53
State-space Model Eduardo Rossi University of Pavia November 2014 Rossi State-space Model Fin. Econometrics - 2014 1 / 53 Outline 1 Motivation 2 Introduction 3 The Kalman filter 4 Forecast errors 5 State
More informationSparse Support Vector Machines by Kernel Discriminant Analysis
Sparse Support Vector Machines by Kernel Discriminant Analysis Kazuki Iwamura and Shigeo Abe Kobe University - Graduate School of Engineering Kobe, Japan Abstract. We discuss sparse support vector machines
More informationNORMAL CHARACTERIZATION BY ZERO CORRELATIONS
J. Aust. Math. Soc. 81 (2006), 351-361 NORMAL CHARACTERIZATION BY ZERO CORRELATIONS EUGENE SENETA B and GABOR J. SZEKELY (Received 7 October 2004; revised 15 June 2005) Communicated by V. Stefanov Abstract
More informationEconomics 620, Lecture 5: exp
1 Economics 620, Lecture 5: The K-Variable Linear Model II Third assumption (Normality): y; q(x; 2 I N ) 1 ) p(y) = (2 2 ) exp (N=2) 1 2 2(y X)0 (y X) where N is the sample size. The log likelihood function
More informationTesting a Normal Covariance Matrix for Small Samples with Monotone Missing Data
Applied Mathematical Sciences, Vol 3, 009, no 54, 695-70 Testing a Normal Covariance Matrix for Small Samples with Monotone Missing Data Evelina Veleva Rousse University A Kanchev Department of Numerical
More informationw 1 output input &N 1 x w n w N =&2
ISSN 98-282 Technical Report L Noise Suppression in Training Data for Improving Generalization Akiko Nakashima, Akira Hirabayashi, and Hidemitsu OGAWA TR96-9 November Department of Computer Science Tokyo
More informationCS281A/Stat241A Lecture 17
CS281A/Stat241A Lecture 17 p. 1/4 CS281A/Stat241A Lecture 17 Factor Analysis and State Space Models Peter Bartlett CS281A/Stat241A Lecture 17 p. 2/4 Key ideas of this lecture Factor Analysis. Recall: Gaussian
More informationThe Multivariate Gaussian Distribution [DRAFT]
The Multivariate Gaussian Distribution DRAFT David S. Rosenberg Abstract This is a collection of a few key and standard results about multivariate Gaussian distributions. I have not included many proofs,
More informationLinear Quadratic Zero-Sum Two-Person Differential Games Pierre Bernhard June 15, 2013
Linear Quadratic Zero-Sum Two-Person Differential Games Pierre Bernhard June 15, 2013 Abstract As in optimal control theory, linear quadratic (LQ) differential games (DG) can be solved, even in high dimension,
More informationChris Bishop s PRML Ch. 8: Graphical Models
Chris Bishop s PRML Ch. 8: Graphical Models January 24, 2008 Introduction Visualize the structure of a probabilistic model Design and motivate new models Insights into the model s properties, in particular
More informationMRAGPC Control of MIMO Processes with Input Constraints and Disturbance
Proceedings of the World Congress on Engineering and Computer Science 9 Vol II WCECS 9, October -, 9, San Francisco, USA MRAGPC Control of MIMO Processes with Input Constraints and Disturbance A. S. Osunleke,
More informationECE521 week 3: 23/26 January 2017
ECE521 week 3: 23/26 January 2017 Outline Probabilistic interpretation of linear regression - Maximum likelihood estimation (MLE) - Maximum a posteriori (MAP) estimation Bias-variance trade-off Linear
More informationA Unified Bayesian Framework for Face Recognition
Appears in the IEEE Signal Processing Society International Conference on Image Processing, ICIP, October 4-7,, Chicago, Illinois, USA A Unified Bayesian Framework for Face Recognition Chengjun Liu and
More informationOPTIMAL CONTROL AND ESTIMATION
OPTIMAL CONTROL AND ESTIMATION Robert F. Stengel Department of Mechanical and Aerospace Engineering Princeton University, Princeton, New Jersey DOVER PUBLICATIONS, INC. New York CONTENTS 1. INTRODUCTION
More informationPhys 201. Matrices and Determinants
Phys 201 Matrices and Determinants 1 1.1 Matrices 1.2 Operations of matrices 1.3 Types of matrices 1.4 Properties of matrices 1.5 Determinants 1.6 Inverse of a 3 3 matrix 2 1.1 Matrices A 2 3 7 =! " 1
More informationSTAT 100C: Linear models
STAT 100C: Linear models Arash A. Amini April 27, 2018 1 / 1 Table of Contents 2 / 1 Linear Algebra Review Read 3.1 and 3.2 from text. 1. Fundamental subspace (rank-nullity, etc.) Im(X ) = ker(x T ) R
More informationGaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012
Gaussian Processes Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 01 Pictorial view of embedding distribution Transform the entire distribution to expected features Feature space Feature
More informationFurther Results on Model Structure Validation for Closed Loop System Identification
Advances in Wireless Communications and etworks 7; 3(5: 57-66 http://www.sciencepublishinggroup.com/j/awcn doi:.648/j.awcn.735. Further esults on Model Structure Validation for Closed Loop System Identification
More informationControlo Switched Systems: Mixing Logic with Differential Equations. João P. Hespanha. University of California at Santa Barbara.
Controlo 00 5 th Portuguese Conference on Automatic Control University of Aveiro,, September 5-7, 5 00 Switched Systems: Mixing Logic with Differential Equations João P. Hespanha University of California
More informationStructured Stochastic Uncertainty
Structured Stochastic Uncertainty Bassam Bamieh Department of Mechanical Engineering University of California at Santa Barbara Santa Barbara, CA, 9306 bamieh@engineeringucsbedu Abstract We consider linear
More informationSystem identification and control with (deep) Gaussian processes. Andreas Damianou
System identification and control with (deep) Gaussian processes Andreas Damianou Department of Computer Science, University of Sheffield, UK MIT, 11 Feb. 2016 Outline Part 1: Introduction Part 2: Gaussian
More informationLecture 6: April 19, 2002
EE596 Pat. Recog. II: Introduction to Graphical Models Spring 2002 Lecturer: Jeff Bilmes Lecture 6: April 19, 2002 University of Washington Dept. of Electrical Engineering Scribe: Huaning Niu,Özgür Çetin
More information