Dynamic Consistency for Stochastic Optimal Control Problems
|
|
- Frank Little
- 5 years ago
- Views:
Transcription
1 Dynamic Consistency for Stochastic Optimal Control Problems Cadarache Summer School CEA/EDF/INRIA 2012 Pierre Carpentier Jean-Philippe Chancelier Michel De Lara SOWG June 2012
2 Lecture outline Introduction Distributed formulation SOC with constraints Reduction to finite-dimensional problem
3 Informal definition of dynamically consistent problems Informal definition For a sequence of dynamic optimization problems, we aim at discussing a notion of consistency over time. At the very first time step t 0, formulate an optimization problem that yields optimal decision rules for all the forthcoming time steps t 0,t 1,...,T; At the next time step t 1, formulate a new optimization problem starting at time t 1 that yields a new sequence of optimal decision rules. This process can be continued until the final time T is reached. A family of optimization problems is said to be dynamically consistent if the optimal strategies obtained when solving the original problem remain optimal for all subsequent problems.
4 Informal definition of dynamically consistent problems Decision at time t 0 State u 0,0 u 1 0,k u 2 0,k u 3 0,k depends on decisions computed at later times for possible future states. t 0 t k T Time
5 Informal definition of dynamically consistent problems Time consistent decision at time t k State u 0,0 t 0 u 0,k t k u k,k T Time Decision u k,k computed using a problem formulated at time t k knowing where I am at time t k or decision u 0,k. computed at time t 0 and to be applied at time t k for the same position should be the same.
6 Informal definition of dynamically consistent problems An example: deterministic shortest path If A B C D E is optimal then C D E is optimal. Arriving at C, If I behave rationally I should not deviate from the originally chosen route starting at A. t=1 B t=2 C E A t=0 D
7 Dynamic Programming and dynamically consistent problems Dynamic Programming and dynamically consistent problems Dynamic Programing gives dynamically consistent controls It gives a family of problems indexed by time: the solution of problem at time t uses the solution of problem at time t +1: V k x = min E [ L k x,u,wk+1 +Vk+1 ftk x,u,w u k+1 ] Choose control at time t k optimizing the sum of a current cost and of a running cost. The running cost has an interpretation as the optimal cost function of a problem at time t k+1.
8 Dynamic Programming and dynamically consistent problems Dynamic Programming and dynamically consistent problems 2 The problem at time t k+1 is the following: V k+1 x = min X,U T 1 E L t Xt,U t,w t+1 +K XT Xtk+1 = x, t=t k+1 s.t. X t+1 = f t Xt,U t,w t+1, t = tk+1,...,t 1, U t X t t = t k+1,...,t 1.
9 Deterministic case Deterministic case State u 0,k u 0,0 u k,k Exact state match at time t k t 0 t k T Time
10 Deterministic case Deterministic case State u 0,k u 0,0 u k,k Independence of the initial condition. t 0 t k T Time
11 Deterministic case Deterministic case State u 0,0 u 0,k u k,k Taking care of all possible initial states at time t k through state feedback. t 0 t k T Time
12 Deterministic case A first example in the deterministic case 1 min u t0,...,u T 1,x t0,...,x T T 1 t=t 0 L t xt,u t +K xt, Dt0 subject to x t+1 = f t xt,u t, xt0 given. Suppose a solution to this problem exists: u t 0,...,u T 1 : controls indexed by time t, the state variable. x t0,xt 1,...,x T : optimal path for No need for more information since the model is deterministic. these controls depend on the hidden parameter x t0, these controls are usually not optimal for x t 0 x t0.
13 Deterministic case A first example in the deterministic case 2 Consider the natural subsequent problems for every t i t 0 : min u ti,...,u T 1,x ti,...,x T T 1 t=t i L t xt,u t +K xt, Dti subject to x t+1 = f t xt,u t, xti given. One makes the two following observations. 1. Independence of the initial condition. In the very particular case where the solution to Problem D ti does not depend on x ti, Problems {D ti } ti are dynamically consistent. 2. True deterministic world. Suppose that the initial condition for Problem D ti is given by xt i = f ti x ti 1,ut i 1 exact model, then Problems {D ti } ti are dynamically consistent. Otherwise, adding disturbances to the problem brings inconsistency.
14 Deterministic case A first example in the deterministic case 3 Solve now Problem D t0 using Dynamic Programming DP: φ t 0,...,φ T 1 : controls depending on both t and x. The following result is a direct application of the DP principle. 3. Right amount of information. Suppose that one is looking for strategies as feedback functions φ t depending on state x. Then Problems {D ti } ti are dynamically consistent. As a first conclusion, time consistency is recovered provided we let the decision rules depend upon a sufficiently rich information.
15 Deterministic case Independence of the initial condition for every t = t 0,...,T 1, functions l t : U t R and g t : U t R, and assume that x t is scalar. Let K be a scalar constant and consider the following deterministic optimal control problem: min x,u T 1 t=t 0 l t u t x t +Kx T, s.t. x t0 given, x t+1 = g t u t x t, t = t 0,...,T 1.
16 Deterministic case Independence of the initial condition 2 Variables x t can be recursively replaced using dynamics g t leading to: min u T 1 t=t 0 l t u t g t 1 u t 1...g t0 u t0 x t0 +Kg T 1 u T 1...g t0 u t0 x t0. The optimal cost is linear with respect to x t0. Suppose that x t0 only takes positive values. Then x t0 has no influence on the minimizer. Remains true at subsequent time steps provided that dynamics are such that x t remains positive for every time step. The dynamic consistency property holds true. This example which may look very special but we will see later on that it is analogous to familiar SOC problems.
17 Deterministic case The guiding principle of the lecture To obtain Dynamical Consistency, let the decision rules depend upon a sufficiently rich information set.
18 Introduction Informal definition of dynamically consistent problems Dynamic Programming and dynamically consistent problems Deterministic case Distributed formulation A classical SOC Problem Equivalent distributed formulation SOC with constraints Constraints typology A toy example with finite probabilities Back to time consistency Reduction to finite-dimensional problem An equivalent problem with added state and control Dynamic programing
19 A classical SOC Problem A classical SOC Problem Control variables U = U t t=t0,...,t 1. Noise variables W = W t t=t1,...,t. Markovian setting: noise variables X t0,w t1,...,w T are independent. The problem S t0 starting at t 0 writes: [ T 1 min X,U E s.t. X t0 given, ] L t Xt,U t,w t+1 +K XT, t=t 0 X t+1 = f t Xt,U t,w t+1, U t X t0,w t1,...,w t, t = t 0,...,T 1, We know that there is no loss of optimality in looking for the optimal strategy as a state feedback control U t = φ t X t. 1
20 A classical SOC Problem Stochastic optimal control: the classical case 2 Can be solved using Dynamic Programming: V T x = Kx, Vtx = min E L t x,u,w u U t+1 +Vt+1 ft x,u,w t+1. It is clear while inspecting the DP equation that optimal strategies {φ t} t t0 remain optimal for the subsequent optimization problems: min U ti,...,u T 1,X ti,...,x T T 1 E L t X t,u t,w t+1 +KX T, t=t i subject to: X ti given, S ti X t+1 = f t X t,u t,w t+1, U t X ti,w ti +1,...,W t.
21 Equivalent distributed formulation An equivalent distributed formulation Ξ t a linear space of R-valued measurable functions on X t. Υ t space of signed measures on X t. We consider a dual pairing between Ξ t and Υ t by considering the bilinear form: ξ,µ = ξxdµx for ξ Ξ t and µ Υ t. X t When X t is a random variable taking values in X t and distributed according to µ t. Then ξ t,µ t = E ξ t X t.
22 Equivalent distributed formulation Fokker-Planck equation: state probability law dynamics For a given feedback laws φ t : X t U t we define T φt t : Ξ t+1 Ξ t : T φ t t ξ t+1 x def = E ξ t+1 X t+1 X t = x x X t. Using state dynamics and Markovian setting we obtain: T φ t t ξ t+1 x = E ξ t+1 ft x,φ t x,w t+1 x X t. Denoting by T φt t the adjoint operator of T φ t t T φ t ξ,µ = ξ, T φ µ t The state probability law driven by the chosen feedback follows: µ t+1 = T φt µt t t = t 0,...,T 1, µ t0 given.
23 Equivalent distributed formulation Revisiting the expected cost Next we introduce the operator Λ φt t : X t R: Λ φt t x def = E L t x,φt x,w t+1 meant to be the expected cost at time t for each possible state value when feedback function φ t is applied. Using Λ φt t we have: E L t Xt,φ t Xt,Wt+1 = E Λ φ t t X t = Λ φt t,µ t.
24 Equivalent distributed formulation Equivalent deterministic infinite-dimensional problem We obtain an equivalent deterministic infinite-dimensional optimal control problem D t0 : min Φ,µ T 1 t=t 0 s.t. µ t0 given, µ t+1 = Λ Φt t,µ t + K,µ T, T Φt t µt, t = t 0,...,T 1, Very similar to the Independence of the initial condition example
25 Equivalent distributed formulation Solving D t0 using Dynamic Programming V T µ = K,µ. A V T 1 µ = min Λ φ φ T 1,µ φ µ +V T T 1. Optimal feedback Γ T 1 : µ φ µ a priori depends on x and µ. V T 1 µ = min Λ φ φ T 1 +Aφ T 1 K,µ, = min Λ φ T 1 φ +Aφ T 1 K xµdx. X
26 Equivalent distributed formulation Solving D t0 using Dynamic Programming 2 Interchanging minimization and expectation operators leads to: optimal Γ T 1 does not depend on µ : Γ T 1 φ T 1, V T 1 again depends on µ in a multiplicative manner.
27 Introduction Informal definition of dynamically consistent problems Dynamic Programming and dynamically consistent problems Deterministic case Distributed formulation A classical SOC Problem Equivalent distributed formulation SOC with constraints Constraints typology A toy example with finite probabilities Back to time consistency Reduction to finite-dimensional problem An equivalent problem with added state and control Dynamic programing
28 Constraints typology Constraints in stochastic optimal control Different kinds of constraints in stochastic optimization: almost-sure constraint : gx T a P-a.s., chance constraint : P gx T a p, expectation constraint : E gx T a,... A chance constraint can be modelled as an expectation constraint: P gx T a = E 1 X adx T, with X ad = { x X, gx a }. Chance constraints bring both theoretical and numerical difficulties, especially convexity [Prékopa, 1995]. However the difficulty we are interested in is common to chance and expectation constraints. In the sequel, we concentrate on adding an expectation constraint.
29 Constraints typology Probability constraint on state trajectories Water level P { X t x t,t [0,T] } a Minimal water level Time
30 Constraints typology Transformed [ ] into a Probability constraints on final state : E g X T a Motivated by a joint probability constraint: P { γ t Xt bt, t = t 0,...,T } a. Introducing a new binary state variable Y t {0,1}: Y t0 = 1, Y t+1 = Y t 1 { γ t+1 X t+1 b t+1 }, E [ Y T ] = 1 P { γt Xt bt, t = t 0,...,T } a
31 Constraints typology Problem setting with expectation constraint Control variables U = U t t=t0,...,t 1. Noise variables W = W t t=t1,...,t. Markovian setting: noises variables X t0,w t1,...,w T are independent. The problem starting at t 0 writes: min X,U [ T 1 ] E L t Xt,U t,w t+1 +K XT, t=t 0 s.t. X t0 given, X t+1 = f t Xt,U t,w t+1, U t X t0,w t1,...,w t, t = t 0,...,T 1, E [ g X T ] a, a R
32 Constraints typology Problem at time t i min X,U [ T 1 ] E L t Xt,U t,w t+1 +K XT, t=t i s.t. X t+1 = f t Xt,U t,w t+1, Xti given, U t X ti,w ti+1,...,w t, t = t i,...,t 1, E [ g X T ] a. Not dynamically consistent with the usual state variable. With new appropriate state variable, one regains dynamical consistency. Optimal strategy at time t is a function of Φ t0,t : X U since information structure is not modified by the last constraint.
33 Constraints typology Equivalent deterministic infinite-dimensional problem We obtain an equivalent deterministic infinite-dimensional optimal control problem. min Φ,µ T 1 t=t 0 s.t. µ t0 given, µ t+1 = Λ Φt t,µ t + K,µ T +χ { g,µ a} µ T T Φt t g,µ T a. µt, t = t 0,...,T 1, The last constraint implies that the optimal feedback laws depend on the initial condition µ t0 and we do not have dynamical consistency.
34 A toy example with finite probabilities Introduction Informal definition of dynamically consistent problems Dynamic Programming and dynamically consistent problems Deterministic case Distributed formulation A classical SOC Problem Equivalent distributed formulation SOC with constraints Constraints typology A toy example with finite probabilities Back to time consistency Reduction to finite-dimensional problem An equivalent problem with added state and control Dynamic programing
35 A toy example with finite probabilities Simple case-study: a discrete controlled Markov chain The state space X = {1,2,3}: Discrete possible values of a reservoir level. x = 1 resp. x = 3 the lower resp. upper level of the reservoir. The control action u takes values in U = {0,1}, the value 1 corresponding to using some given water release to produce some electricity. The noise variable: stochastic inflow takes its values in W = { 1,0,1}. The discrete probability law of each noise variable W t being characterized by the weights {, 0, +}. Finally, the reservoir dynamics is: X t+1 = min x,maxx,x t U t +W t+1.
36 A toy example with finite probabilities Simple case-study: a discrete controlled Markov chain Let µ t be the discrete probability law associated with the state random variable X t, the initial probability law µ t0 being given. In such a discrete case, it is easy to compute the transition matrix P u giving the Markov chain transitions for each possible value u of the control, with the following interpretation: P u ij = P X t+1 = j X t = i,u t = u = P {min x,maxx,i u +Wt+1 = j }.
37 A toy example with finite probabilities Simple case-study: a discrete controlled Markov chain We obtain for the reservoir problem: P 0 = 0 +, P 1 = and we denote by P u i the i th row of matrix P u.
38 A toy example with finite probabilities Simple case-study: a discrete controlled Markov chain Let φ t : X U be a feedback law at time t. We denote by Φ t the set of admissible feedbacks at time t. In our discrete case, cardφ t = cardu cardx = 8 for all t. The transition matrix P φt associated with such a feedback is obtained by properly selecting rows of the transition matrices P u, namely: P φt1 1 P φt = P φt2 2. P φt3 3 Then the dynamics given by the Fokker-Planck equation writes: µ t+1 = P φt µt, and the state µ t involved in this dynamic equation is a three-dimensional column vector µ t = µ 1,t,µ 2,t,µ 3,t
39 A toy example with finite probabilities The cost at time t is supposed to be linear, equal to p t U t, p t being a negative deterministic price. No final cost K 0. The reservoir level at final time T must be equal to x with a probability level at least equal to π. The distributed formulation associated with the reservoir control problem is: min {φ t Φ t} T 1 t=t 0 T 1 t=t 0 p t φt,µ t, 6a subject to: µ t+1 = P φt µt, µ t0 given, 6b 1{x},µ T π, 6c with 1 {x} x = 1 if x = x, and 0 otherwise.
40 A toy example with finite probabilities The previous problem can be solved using DP: the state equation is a two-dimensional µ 1,t +µ 2,t +µ 3,t = 1 Suppose now that the number of reservoir discrete levels is n rather than 3, with n big enough: the resolution of the distributed formulation suffers from the curse of dimensionality n 1-dimensional state, The ultimate curse being to consider that the level takes values in the interval [x,x], so that the µ t s are continuous probability laws.
41 Back to time consistency Back to Dynamical Consistency We can use Dynamic Programing: Let V t µ t be the optimal cost of the problem starting at time t with initial condition µ t. { K,µ if g,µ a, V T µ = + otherwise, and, for every t = t 0,...,T 1 and every probability law µ on X: V t µ = min Λ Φt t,µ +V t+1 Φ t T Φt t µ. Optimal feedback functions Φ t depend on µ t. Consistency recovered for strategies depending on X t and µ t.
42 Back to time consistency Back to Dynamical Consistency 2 Optimal feedback functions Φ t depend on µ t µ min Λ Φt t,µ +V t+1 T Φt t. Φ t But it also depends on X t because of the distributed formulation U t = φ t X t! Consistency recovered for strategies depending on X t and µ t. The new state variable to consider X t,µ t is an infinite dimensional object!
43 Introduction Informal definition of dynamically consistent problems Dynamic Programming and dynamically consistent problems Deterministic case Distributed formulation A classical SOC Problem Equivalent distributed formulation SOC with constraints Constraints typology A toy example with finite probabilities Back to time consistency Reduction to finite-dimensional problem An equivalent problem with added state and control Dynamic programing
44 An equivalent problem with added state and control Equivalent problem: added state process Z and control V min U,V,X,Z T 1 E L t X t,u t,w t+1 +KX T, t=t 0 X t0 = x t0, X t+1 = f t X t,u t,w t+1, Z t0 = 0, Z t+1 = Z t +V t, Almost sure final constraint: gx T Z T a.. Measurability constraints: U t F t, V t F t+1. Additional time constraints: E V t F t = 0.
45 An equivalent problem with added state and control Equivalent problem 2 Let U t0,...,u T 1 and X t0,...,x T satisfying the constraints of Initial problem. We define V t0,...,v T 1 and Z t0,...,z T as follow: V t = E gx T F t+1 E gxt F t, Zt+1 = Z t +V t. Noting that the σ-field F t0 is the minimal σ-field and that the random variable X T is F T -measurable, we have: Z T = gx T E gx T. Hence, U, X, V and Z satisfy the set of constraints of the new problem.
46 An equivalent problem with added state and control Equivalent problem 3 Let U t0,...,u T 1, X t0,...,x T, V t0,...,v T 1 and Z t0,...,z T satisfying the constraints of V,Z-problem be given. Then, Z T = T 1 t=t 0 V t. Thus we have: E T 1 Z T = E E V t F t = 0. t=t 0 If follows that gx T Z T a E gx T a, and therefore the processes U t0,...,u T 1 and X t0,...,x T satisfy the constraints of Initial problem To conclude, since the two problems share the same sets as admissible constraints and share the same criteria to optimize, they are equivalent.
47 An equivalent problem with added state and control Equivalent problem 4 1. Dynamic Programming will give feedbacks as functions of X t,z t. 2. The new problem is more intricate: added state and added control, Vt depends on W t+1 Hazard Decision some new constraints on the controls are to be taken into account. 3. E gx T F t : perception of the risk constraint at time t. 4. V t = E gx T F t+1 E gxt F t : variation between time t and time t +1 of this perception. 5. Z t: cumulative variation over time of the risk constraint.
48 Dynamic programing Dynamic programing In order to use Dynamic programming on problem 7, we will first recall on a simplified model how Dynamic Programming works on a classical SOP. min X,U E ] L t Xt,U t,w t+1 +K XT, t=t 0 [ T 1 s.t. X t0 given, X t+1 = f t Xt,U t,w t+1, U t F t t = t 0,...,T 1, Then we will show the modifications induced by hazard-decision control or E V t F t = 0.
49 Dynamic programing Dynamic programing steps in classical framework 1. Rewrites criterion using conditional expectations: E E L 0 X 0,U 0 +E L 1 X 1,U 1 +KX 2 F 1 F0 2. minimization with respect to U t and X t enters up to the conditional expectation with respect to F t. 3. Recursive computation from the most internal problem to the outer one t = t 0. The most internal t = T 1: min E hx T 1,U T 1,W T F T 1. U F T 1 T 1 hx,u,w = L T 1 x,u+k f T 1 x,u,w
50 Dynamic programing Hazard Decision framework Hazard Decision: U t F t+1 Interchange of expectation and minimization can be done one step further. E min hx T 1,U T 1,W T F T 1. U F T 1 T with hx,u,w = L T 1 x,u,w+k f T 1 x,u,w Optimal feedback U T 1 can be chosen as a X T 1,W T -measurable function.
51 Dynamic programing Hazard Decision framework with constraint Control constraint: E U t F t = 0. We are led to the following problem at time t = T 1: min E hx T 1,U T 1,W T F T 1, U F T 1 T subject to: E U T 1 F T 1 = 0. Optimal feedback U T 1 can still be chosen as a X T 1,W T -measurable function? Yes and the proof is actually available when F T 1 is generated by finite valued random variables.
52 Dynamic programing Back to the equivalent problem The Hazard-Decision V and Decision-Hazard U controls. Initialization of the Bellman function at time T: V T x,z = Kx+χ G ad x,z, χ G ad characteristic function of Gad = { x,z gx z a } ; Bellman function at time t: V t x,z = min u U min E hx,u,z,v,w V W t+1, t+1 subject to: E V = 0. hx,u,z,v,w def = L t x,u,w+v t+1 ft x,u,w,z +v U t X t,z t and V t X t,z t,w t+1.
arxiv: v1 [math.oc] 20 May 2010
DYNAMIC CONSISTENCY FOR STOCHASTIC OPTIMAL CONTROL PROBLEMS PIERRE CARPENTIER, JEAN-PHILIPPE CHANCELIER, GUY COHEN, MICHEL DE LARA, AND PIERRE GIRARDEAU arxiv:1005.3605v1 [math.oc] 20 May 2010 Abstract.
More informationDeterministic Dynamic Programming
Deterministic Dynamic Programming 1 Value Function Consider the following optimal control problem in Mayer s form: V (t 0, x 0 ) = inf u U J(t 1, x(t 1 )) (1) subject to ẋ(t) = f(t, x(t), u(t)), x(t 0
More informationExam 2, Solution 4. Jordan Paschke
Exam 2, Solution 4 Jordan Paschke Problem 4. Suppose that y 1 (t) = t and y 2 (t) = t 3 are solutions to the equation 3t 3 y t 2 y 6ty + 6y = 0, t > 0. ( ) Find the general solution of the above equation.
More information7. Shortest Path Problems and Deterministic Finite State Systems
7. Shortest Path Problems and Deterministic Finite State Systems In the next two lectures we will look at shortest path problems, where the objective is to find the shortest path from a start node to an
More informationFenchel-Moreau Conjugates of Inf-Transforms and Application to Stochastic Bellman Equation
Fenchel-Moreau Conjugates of Inf-Transforms and Application to Stochastic Bellman Equation Jean-Philippe Chancelier and Michel De Lara CERMICS, École des Ponts ParisTech First Conference on Discrete Optimization
More informationAlternative Characterization of Ergodicity for Doubly Stochastic Chains
Alternative Characterization of Ergodicity for Doubly Stochastic Chains Behrouz Touri and Angelia Nedić Abstract In this paper we discuss the ergodicity of stochastic and doubly stochastic chains. We define
More informationKalman Filter and Parameter Identification. Florian Herzog
Kalman Filter and Parameter Identification Florian Herzog 2013 Continuous-time Kalman Filter In this chapter, we shall use stochastic processes with independent increments w 1 (.) and w 2 (.) at the input
More informationIntroduction - Motivation. Many phenomena (physical, chemical, biological, etc.) are model by differential equations. f f(x + h) f(x) (x) = lim
Introduction - Motivation Many phenomena (physical, chemical, biological, etc.) are model by differential equations. Recall the definition of the derivative of f(x) f f(x + h) f(x) (x) = lim. h 0 h Its
More information1 Markov decision processes
2.997 Decision-Making in Large-Scale Systems February 4 MI, Spring 2004 Handout #1 Lecture Note 1 1 Markov decision processes In this class we will study discrete-time stochastic systems. We can describe
More informationStochastic and Adaptive Optimal Control
Stochastic and Adaptive Optimal Control Robert Stengel Optimal Control and Estimation, MAE 546 Princeton University, 2018! Nonlinear systems with random inputs and perfect measurements! Stochastic neighboring-optimal
More information4th Preparation Sheet - Solutions
Prof. Dr. Rainer Dahlhaus Probability Theory Summer term 017 4th Preparation Sheet - Solutions Remark: Throughout the exercise sheet we use the two equivalent definitions of separability of a metric space
More informationPositive Markov Jump Linear Systems (PMJLS) with applications
Positive Markov Jump Linear Systems (PMJLS) with applications P. Bolzern, P. Colaneri DEIB, Politecnico di Milano - Italy December 12, 2015 Summary Positive Markov Jump Linear Systems Mean stability Input-output
More informationENSC327 Communications Systems 19: Random Processes. Jie Liang School of Engineering Science Simon Fraser University
ENSC327 Communications Systems 19: Random Processes Jie Liang School of Engineering Science Simon Fraser University 1 Outline Random processes Stationary random processes Autocorrelation of random processes
More informationHJB equations. Seminar in Stochastic Modelling in Economics and Finance January 10, 2011
Department of Probability and Mathematical Statistics Faculty of Mathematics and Physics, Charles University in Prague petrasek@karlin.mff.cuni.cz Seminar in Stochastic Modelling in Economics and Finance
More informationGame Theory Extra Lecture 1 (BoB)
Game Theory 2014 Extra Lecture 1 (BoB) Differential games Tools from optimal control Dynamic programming Hamilton-Jacobi-Bellman-Isaacs equation Zerosum linear quadratic games and H control Baser/Olsder,
More informationSolution of Stochastic Optimal Control Problems and Financial Applications
Journal of Mathematical Extension Vol. 11, No. 4, (2017), 27-44 ISSN: 1735-8299 URL: http://www.ijmex.com Solution of Stochastic Optimal Control Problems and Financial Applications 2 Mat B. Kafash 1 Faculty
More informationStochastic Shortest Path Problems
Chapter 8 Stochastic Shortest Path Problems 1 In this chapter, we study a stochastic version of the shortest path problem of chapter 2, where only probabilities of transitions along different arcs can
More informationCHAPTER 2 THE MAXIMUM PRINCIPLE: CONTINUOUS TIME. Chapter2 p. 1/67
CHAPTER 2 THE MAXIMUM PRINCIPLE: CONTINUOUS TIME Chapter2 p. 1/67 THE MAXIMUM PRINCIPLE: CONTINUOUS TIME Main Purpose: Introduce the maximum principle as a necessary condition to be satisfied by any optimal
More informationStatic and Dynamic Optimization (42111)
Static and Dynamic Optimization (421) Niels Kjølstad Poulsen Build. 0b, room 01 Section for Dynamical Systems Dept. of Applied Mathematics and Computer Science The Technical University of Denmark Email:
More informationOn the Convergence of Optimistic Policy Iteration
Journal of Machine Learning Research 3 (2002) 59 72 Submitted 10/01; Published 7/02 On the Convergence of Optimistic Policy Iteration John N. Tsitsiklis LIDS, Room 35-209 Massachusetts Institute of Technology
More informationMatthew Zyskowski 1 Quanyan Zhu 2
Matthew Zyskowski 1 Quanyan Zhu 2 1 Decision Science, Credit Risk Office Barclaycard US 2 Department of Electrical Engineering Princeton University Outline Outline I Modern control theory with game-theoretic
More informationQUALIFYING EXAM IN SYSTEMS ENGINEERING
QUALIFYING EXAM IN SYSTEMS ENGINEERING Written Exam: MAY 23, 2017, 9:00AM to 1:00PM, EMB 105 Oral Exam: May 25 or 26, 2017 Time/Location TBA (~1 hour per student) CLOSED BOOK, NO CHEAT SHEETS BASIC SCIENTIFIC
More informationUncertainty quantification and systemic risk
Uncertainty quantification and systemic risk Josselin Garnier (Université Paris Diderot) with George Papanicolaou and Tzu-Wei Yang (Stanford University) February 3, 2016 Modeling systemic risk We consider
More informationGaussian, Markov and stationary processes
Gaussian, Markov and stationary processes Gonzalo Mateos Dept. of ECE and Goergen Institute for Data Science University of Rochester gmateosb@ece.rochester.edu http://www.ece.rochester.edu/~gmateosb/ November
More information14 - Gaussian Stochastic Processes
14-1 Gaussian Stochastic Processes S. Lall, Stanford 211.2.24.1 14 - Gaussian Stochastic Processes Linear systems driven by IID noise Evolution of mean and covariance Example: mass-spring system Steady-state
More informationEN Applied Optimal Control Lecture 8: Dynamic Programming October 10, 2018
EN530.603 Applied Optimal Control Lecture 8: Dynamic Programming October 0, 08 Lecturer: Marin Kobilarov Dynamic Programming (DP) is conerned with the computation of an optimal policy, i.e. an optimal
More informationLecture 6: Bayesian Inference in SDE Models
Lecture 6: Bayesian Inference in SDE Models Bayesian Filtering and Smoothing Point of View Simo Särkkä Aalto University Simo Särkkä (Aalto) Lecture 6: Bayesian Inference in SDEs 1 / 45 Contents 1 SDEs
More informationMGMT 69000: Topics in High-dimensional Data Analysis Falll 2016
MGMT 69000: Topics in High-dimensional Data Analysis Falll 2016 Lecture 14: Information Theoretic Methods Lecturer: Jiaming Xu Scribe: Hilda Ibriga, Adarsh Barik, December 02, 2016 Outline f-divergence
More informationStochastic Composition Optimization
Stochastic Composition Optimization Algorithms and Sample Complexities Mengdi Wang Joint works with Ethan X. Fang, Han Liu, and Ji Liu ORFE@Princeton ICCOPT, Tokyo, August 8-11, 2016 1 / 24 Collaborators
More informationDecomposability and time consistency of risk averse multistage programs
Decomposability and time consistency of risk averse multistage programs arxiv:1806.01497v1 [math.oc] 5 Jun 2018 A. Shapiro School of Industrial and Systems Engineering Georgia Institute of Technology Atlanta,
More informationZ i Q ij Z j. J(x, φ; U) = X T φ(t ) 2 h + where h k k, H(t) k k and R(t) r r are nonnegative definite matrices (R(t) is uniformly in t nonsingular).
2. LINEAR QUADRATIC DETERMINISTIC PROBLEM Notations: For a vector Z, Z = Z, Z is the Euclidean norm here Z, Z = i Z2 i is the inner product; For a vector Z and nonnegative definite matrix Q, Z Q = Z, QZ
More informationOptimal Control. McGill COMP 765 Oct 3 rd, 2017
Optimal Control McGill COMP 765 Oct 3 rd, 2017 Classical Control Quiz Question 1: Can a PID controller be used to balance an inverted pendulum: A) That starts upright? B) That must be swung-up (perhaps
More informationThe Social Cost of Stochastic & Irreversible Climate Change
The Social Cost of Stochastic & Irreversible Climate Change Thomas S. Lontzek (University of Zurich and RDCEP) joint work with Yongyang Cai and Kenneth L. Judd (both at Hoover Institution, RDCEP and NBER)
More information2.3 Terminology for Systems of Linear Equations
page 133 e 2t sin 2t 44 A(t) = t 2 5 te t, a = 0, b = 1 sec 2 t 3t sin t 45 The matrix function A(t) in Problem 39, with a = 0 and b = 1 Integration of matrix functions given in the text was done with
More informationLinear Quadratic Zero-Sum Two-Person Differential Games Pierre Bernhard June 15, 2013
Linear Quadratic Zero-Sum Two-Person Differential Games Pierre Bernhard June 15, 2013 Abstract As in optimal control theory, linear quadratic (LQ) differential games (DG) can be solved, even in high dimension,
More informationAn Uncertain Control Model with Application to. Production-Inventory System
An Uncertain Control Model with Application to Production-Inventory System Kai Yao 1, Zhongfeng Qin 2 1 Department of Mathematical Sciences, Tsinghua University, Beijing 100084, China 2 School of Economics
More informationLECTURE 3. Last time:
LECTURE 3 Last time: Mutual Information. Convexity and concavity Jensen s inequality Information Inequality Data processing theorem Fano s Inequality Lecture outline Stochastic processes, Entropy rate
More informationUniformly Uniformly-ergodic Markov chains and BSDEs
Uniformly Uniformly-ergodic Markov chains and BSDEs Samuel N. Cohen Mathematical Institute, University of Oxford (Based on joint work with Ying Hu, Robert Elliott, Lukas Szpruch) Centre Henri Lebesgue,
More informationMarkov Chain Monte Carlo Methods for Stochastic
Markov Chain Monte Carlo Methods for Stochastic Optimization i John R. Birge The University of Chicago Booth School of Business Joint work with Nicholas Polson, Chicago Booth. JRBirge U Florida, Nov 2013
More informationMarkov Chain BSDEs and risk averse networks
Markov Chain BSDEs and risk averse networks Samuel N. Cohen Mathematical Institute, University of Oxford (Based on joint work with Ying Hu, Robert Elliott, Lukas Szpruch) 2nd Young Researchers in BSDEs
More informationRegion of attraction approximations for polynomial dynamical systems
Region of attraction approximations for polynomial dynamical systems Milan Korda EPFL Lausanne Didier Henrion LAAS-CNRS Toulouse & CTU Prague Colin N. Jones EPFL Lausanne Region of Attraction (ROA) ẋ(t)
More informationExtended Formulations, Lagrangian Relaxation, & Column Generation: tackling large scale applications
Extended Formulations, Lagrangian Relaxation, & Column Generation: tackling large scale applications François Vanderbeck University of Bordeaux INRIA Bordeaux-Sud-Ouest part : Defining Extended Formulations
More informationExercises. T 2T. e ita φ(t)dt.
Exercises. Set #. Construct an example of a sequence of probability measures P n on R which converge weakly to a probability measure P but so that the first moments m,n = xdp n do not converge to m = xdp.
More informationLecture 5 Linear Quadratic Stochastic Control
EE363 Winter 2008-09 Lecture 5 Linear Quadratic Stochastic Control linear-quadratic stochastic control problem solution via dynamic programming 5 1 Linear stochastic system linear dynamical system, over
More informationModeling and Analysis of Dynamic Systems
Modeling and Analysis of Dynamic Systems Dr. Guillaume Ducard Fall 2017 Institute for Dynamic Systems and Control ETH Zurich, Switzerland G. Ducard c 1 / 57 Outline 1 Lecture 13: Linear System - Stability
More informationHamilton-Jacobi-Bellman Equation Feb 25, 2008
Hamilton-Jacobi-Bellman Equation Feb 25, 2008 What is it? The Hamilton-Jacobi-Bellman (HJB) equation is the continuous-time analog to the discrete deterministic dynamic programming algorithm Discrete VS
More informationDiscrete-Time Finite-Horizon Optimal ALM Problem with Regime-Switching for DB Pension Plan
Applied Mathematical Sciences, Vol. 10, 2016, no. 33, 1643-1652 HIKARI Ltd, www.m-hikari.com http://dx.doi.org/10.12988/ams.2016.6383 Discrete-Time Finite-Horizon Optimal ALM Problem with Regime-Switching
More informationLMI Methods in Optimal and Robust Control
LMI Methods in Optimal and Robust Control Matthew M. Peet Arizona State University Lecture 15: Nonlinear Systems and Lyapunov Functions Overview Our next goal is to extend LMI s and optimization to nonlinear
More informationModule 05 Introduction to Optimal Control
Module 05 Introduction to Optimal Control Ahmad F. Taha EE 5243: Introduction to Cyber-Physical Systems Email: ahmad.taha@utsa.edu Webpage: http://engineering.utsa.edu/ taha/index.html October 8, 2015
More informationFiltrations, Markov Processes and Martingales. Lectures on Lévy Processes and Stochastic Calculus, Braunschweig, Lecture 3: The Lévy-Itô Decomposition
Filtrations, Markov Processes and Martingales Lectures on Lévy Processes and Stochastic Calculus, Braunschweig, Lecture 3: The Lévy-Itô Decomposition David pplebaum Probability and Statistics Department,
More informationACM/CMS 107 Linear Analysis & Applications Fall 2016 Assignment 4: Linear ODEs and Control Theory Due: 5th December 2016
ACM/CMS 17 Linear Analysis & Applications Fall 216 Assignment 4: Linear ODEs and Control Theory Due: 5th December 216 Introduction Systems of ordinary differential equations (ODEs) can be used to describe
More informationChain differentials with an application to the mathematical fear operator
Chain differentials with an application to the mathematical fear operator Pierre Bernhard I3S, University of Nice Sophia Antipolis and CNRS, ESSI, B.P. 145, 06903 Sophia Antipolis cedex, France January
More informationFormula Sheet for Optimal Control
Formula Sheet for Optimal Control Division of Optimization and Systems Theory Royal Institute of Technology 144 Stockholm, Sweden 23 December 1, 29 1 Dynamic Programming 11 Discrete Dynamic Programming
More informationGame Theory with Information: Introducing the Witsenhausen Intrinsic Model
Game Theory with Information: Introducing the Witsenhausen Intrinsic Model Michel De Lara and Benjamin Heymann Cermics, École des Ponts ParisTech France École des Ponts ParisTech March 15, 2017 Information
More information1 Stochastic Dynamic Programming
1 Stochastic Dynamic Programming Formally, a stochastic dynamic program has the same components as a deterministic one; the only modification is to the state transition equation. When events in the future
More information16.30 Estimation and Control of Aerospace Systems
16.30 Estimation and Control of Aerospace Systems Topic 5 addendum: Signals and Systems Aeronautics and Astronautics Massachusetts Institute of Technology Fall 2010 (MIT) Topic 5 addendum: Signals, Systems
More informationNonlinear Observers. Jaime A. Moreno. Eléctrica y Computación Instituto de Ingeniería Universidad Nacional Autónoma de México
Nonlinear Observers Jaime A. Moreno JMorenoP@ii.unam.mx Eléctrica y Computación Instituto de Ingeniería Universidad Nacional Autónoma de México XVI Congreso Latinoamericano de Control Automático October
More informationLinear-Quadratic-Gaussian (LQG) Controllers and Kalman Filters
Linear-Quadratic-Gaussian (LQG) Controllers and Kalman Filters Emo Todorov Applied Mathematics and Computer Science & Engineering University of Washington Winter 204 Emo Todorov (UW) AMATH/CSE 579, Winter
More informationManagement of dam systems via optimal price control
Available online at www.sciencedirect.com Procedia Computer Science 4 (211) 1373 1382 International Conference on Computational Science, ICCS 211 Management of dam systems via optimal price control Boris
More informationHomogeneous Linear Systems and Their General Solutions
37 Homogeneous Linear Systems and Their General Solutions We are now going to restrict our attention further to the standard first-order systems of differential equations that are linear, with particular
More informationMath 249B. Nilpotence of connected solvable groups
Math 249B. Nilpotence of connected solvable groups 1. Motivation and examples In abstract group theory, the descending central series {C i (G)} of a group G is defined recursively by C 0 (G) = G and C
More informationBrownian Motion. 1 Definition Brownian Motion Wiener measure... 3
Brownian Motion Contents 1 Definition 2 1.1 Brownian Motion................................. 2 1.2 Wiener measure.................................. 3 2 Construction 4 2.1 Gaussian process.................................
More informationMarkov decision processes and interval Markov chains: exploiting the connection
Markov decision processes and interval Markov chains: exploiting the connection Mingmei Teo Supervisors: Prof. Nigel Bean, Dr Joshua Ross University of Adelaide July 10, 2013 Intervals and interval arithmetic
More informationThe first order quasi-linear PDEs
Chapter 2 The first order quasi-linear PDEs The first order quasi-linear PDEs have the following general form: F (x, u, Du) = 0, (2.1) where x = (x 1, x 2,, x 3 ) R n, u = u(x), Du is the gradient of u.
More information6.231 DYNAMIC PROGRAMMING LECTURE 13 LECTURE OUTLINE
6.231 DYNAMIC PROGRAMMING LECTURE 13 LECTURE OUTLINE Control of continuous-time Markov chains Semi-Markov problems Problem formulation Equivalence to discretetime problems Discounted problems Average cost
More informationMATH 220 solution to homework 5
MATH 220 solution to homework 5 Problem. (i Define E(t = k(t + p(t = then E (t = 2 = 2 = 2 u t u tt + u x u xt dx u 2 t + u 2 x dx, u t u xx + u x u xt dx x [u tu x ] dx. Because f and g are compactly
More information3 Compact Operators, Generalized Inverse, Best- Approximate Solution
3 Compact Operators, Generalized Inverse, Best- Approximate Solution As we have already heard in the lecture a mathematical problem is well - posed in the sense of Hadamard if the following properties
More informationMEAN FIELD GAMES WITH MAJOR AND MINOR PLAYERS
MEAN FIELD GAMES WITH MAJOR AND MINOR PLAYERS René Carmona Department of Operations Research & Financial Engineering PACM Princeton University CEMRACS - Luminy, July 17, 217 MFG WITH MAJOR AND MINOR PLAYERS
More informationLecture 4: Numerical Solution of SDEs, Itô Taylor Series, Gaussian Process Approximations
Lecture 4: Numerical Solution of SDEs, Itô Taylor Series, Gaussian Process Approximations Simo Särkkä Aalto University Tampere University of Technology Lappeenranta University of Technology Finland November
More informationCONTROL SYSTEMS, ROBOTICS AND AUTOMATION Vol. XI Stochastic Stability - H.J. Kushner
STOCHASTIC STABILITY H.J. Kushner Applied Mathematics, Brown University, Providence, RI, USA. Keywords: stability, stochastic stability, random perturbations, Markov systems, robustness, perturbed systems,
More informationRobustness and bootstrap techniques in portfolio efficiency tests
Robustness and bootstrap techniques in portfolio efficiency tests Dept. of Probability and Mathematical Statistics, Charles University, Prague, Czech Republic July 8, 2013 Motivation Portfolio selection
More informationAdvanced Computer Networks Lecture 2. Markov Processes
Advanced Computer Networks Lecture 2. Markov Processes Husheng Li Min Kao Department of Electrical Engineering and Computer Science University of Tennessee, Knoxville Spring, 2016 1/28 Outline 2/28 1 Definition
More informationConservation Laws: Systematic Construction, Noether s Theorem, Applications, and Symbolic Computations.
Conservation Laws: Systematic Construction, Noether s Theorem, Applications, and Symbolic Computations. Alexey Shevyakov Department of Mathematics and Statistics, University of Saskatchewan, Saskatoon,
More informationChapter 0 of Calculus ++, Differential calculus with several variables
Chapter of Calculus ++, Differential calculus with several variables Background material by Eric A Carlen Professor of Mathematics Georgia Tech Spring 6 c 6 by the author, all rights reserved - Table of
More information6.241 Dynamic Systems and Control
6.241 Dynamic Systems and Control Lecture 7: State-space Models Readings: DDV, Chapters 7,8 Emilio Frazzoli Aeronautics and Astronautics Massachusetts Institute of Technology February 25, 2011 E. Frazzoli
More informationStochastic optimal control with rough paths
Stochastic optimal control with rough paths Paul Gassiat TU Berlin Stochastic processes and their statistics in Finance, Okinawa, October 28, 2013 Joint work with Joscha Diehl and Peter Friz Introduction
More informationLecture 4: Numerical Solution of SDEs, Itô Taylor Series, Gaussian Approximations
Lecture 4: Numerical Solution of SDEs, Itô Taylor Series, Gaussian Approximations Simo Särkkä Aalto University, Finland November 18, 2014 Simo Särkkä (Aalto) Lecture 4: Numerical Solution of SDEs November
More informationDEPARTMENT OF MATHEMATICS
DEPARTMENT OF MATHEMATICS Ma322 - Final Exam Spring 2011 May 3,4, 2011 DO NOT TURN THIS PAGE UNTIL YOU ARE INSTRUCTED TO DO SO. Be sure to show all work and justify your answers. There are 8 problems and
More informationChap. 3. Controlled Systems, Controllability
Chap. 3. Controlled Systems, Controllability 1. Controllability of Linear Systems 1.1. Kalman s Criterion Consider the linear system ẋ = Ax + Bu where x R n : state vector and u R m : input vector. A :
More informationKernels for Dynamic Textures
Kernels for Dynamic Textures S.V.N. Vishwanathan SVN.Vishwanathan@nicta.com.au http://web.anu.edu.au/~vishy National ICT Australia and Australian National University Joint work with Alex Smola and René
More informationLinear Algebra Review (Course Notes for Math 308H - Spring 2016)
Linear Algebra Review (Course Notes for Math 308H - Spring 2016) Dr. Michael S. Pilant February 12, 2016 1 Background: We begin with one of the most fundamental notions in R 2, distance. Letting (x 1,
More informationAC&ST AUTOMATIC CONTROL AND SYSTEM THEORY SYSTEMS AND MODELS. Claudio Melchiorri
C. Melchiorri (DEI) Automatic Control & System Theory 1 AUTOMATIC CONTROL AND SYSTEM THEORY SYSTEMS AND MODELS Claudio Melchiorri Dipartimento di Ingegneria dell Energia Elettrica e dell Informazione (DEI)
More informationLinear-Quadratic Optimal Control: Full-State Feedback
Chapter 4 Linear-Quadratic Optimal Control: Full-State Feedback 1 Linear quadratic optimization is a basic method for designing controllers for linear (and often nonlinear) dynamical systems and is actually
More information6.231 DYNAMIC PROGRAMMING LECTURE 6 LECTURE OUTLINE
6.231 DYNAMIC PROGRAMMING LECTURE 6 LECTURE OUTLINE Review of Q-factors and Bellman equations for Q-factors VI and PI for Q-factors Q-learning - Combination of VI and sampling Q-learning and cost function
More informationChapter 7. Canonical Forms. 7.1 Eigenvalues and Eigenvectors
Chapter 7 Canonical Forms 7.1 Eigenvalues and Eigenvectors Definition 7.1.1. Let V be a vector space over the field F and let T be a linear operator on V. An eigenvalue of T is a scalar λ F such that there
More informationMATH The Chain Rule Fall 2016 A vector function of a vector variable is a function F: R n R m. In practice, if x 1, x n is the input,
MATH 20550 The Chain Rule Fall 2016 A vector function of a vector variable is a function F: R n R m. In practice, if x 1, x n is the input, F(x 1,, x n ) F 1 (x 1,, x n ),, F m (x 1,, x n ) where each
More informationControl Theory in Physics and other Fields of Science
Michael Schulz Control Theory in Physics and other Fields of Science Concepts, Tools, and Applications With 46 Figures Sprin ger 1 Introduction 1 1.1 The Aim of Control Theory 1 1.2 Dynamic State of Classical
More informationSequential State Estimation (Crassidas and Junkins, Chapter 5)
Sequential State Estimation (Crassidas and Junkins, Chapter 5) Please read: 5.1, 5.3-5.6 5.3 The Discrete-Time Kalman Filter The discrete-time Kalman filter is used when the dynamics and measurements are
More informationLinear Programming Methods
Chapter 11 Linear Programming Methods 1 In this chapter we consider the linear programming approach to dynamic programming. First, Bellman s equation can be reformulated as a linear program whose solution
More informationUncertainty quantification and systemic risk
Uncertainty quantification and systemic risk Josselin Garnier (Université Paris 7) with George Papanicolaou and Tzu-Wei Yang (Stanford University) April 9, 2013 Modeling Systemic Risk We consider evolving
More informationCentralized Versus Decentralized Control - A Solvable Stylized Model in Transportation
Centralized Versus Decentralized Control - A Solvable Stylized Model in Transportation M.-O. Hongler, O. Gallay, R. Colmorn, P. Cordes and M. Hülsmann Ecole Polytechnique Fédérale de Lausanne (EPFL), (CH)
More informationIntroduction to Nonlinear Control Lecture # 3 Time-Varying and Perturbed Systems
p. 1/5 Introduction to Nonlinear Control Lecture # 3 Time-Varying and Perturbed Systems p. 2/5 Time-varying Systems ẋ = f(t, x) f(t, x) is piecewise continuous in t and locally Lipschitz in x for all t
More informationUC Berkeley Summer Undergraduate Research Program 2015 July 8 Lecture
UC Berkeley Summer Undergraduate Research Program 25 July 8 Lecture This lecture is intended to tie up some (potential) loose ends that we have encountered on the road during the past couple of weeks We
More informationProf. Erhan Bayraktar (University of Michigan)
September 17, 2012 KAP 414 2:15 PM- 3:15 PM Prof. (University of Michigan) Abstract: We consider a zero-sum stochastic differential controller-and-stopper game in which the state process is a controlled
More informationMean-Field Games with non-convex Hamiltonian
Mean-Field Games with non-convex Hamiltonian Martino Bardi Dipartimento di Matematica "Tullio Levi-Civita" Università di Padova Workshop "Optimal Control and MFGs" Pavia, September 19 21, 2018 Martino
More informationUtility Maximization in Hidden Regime-Switching Markets with Default Risk
Utility Maximization in Hidden Regime-Switching Markets with Default Risk José E. Figueroa-López Department of Mathematics and Statistics Washington University in St. Louis figueroa-lopez@wustl.edu pages.wustl.edu/figueroa
More information6 Reinforcement Learning
6 Reinforcement Learning As discussed above, a basic form of supervised learning is function approximation, relating input vectors to output vectors, or, more generally, finding density functions p(y,
More informationAbstrct. In this paper, we consider the problem of optimal flow control for a production system with one machine which is subject to failures and prod
Monotonicity of Optimal Flow Control for Failure Prone Production Systems 1 Jian-Qiang Hu 2 and Dong Xiang 3 Communicated By W.B. Gong 1 This work is partially supported by the national Science Foundation
More informationCentralized Versus Decentralized Control - A Solvable Stylized Model in Transportation Logistics
Centralized Versus Decentralized Control - A Solvable Stylized Model in Transportation Logistics O. Gallay, M.-O. Hongler, R. Colmorn, P. Cordes and M. Hülsmann Ecole Polytechnique Fédérale de Lausanne
More informationLECTURES 2-3 : Stochastic Processes, Autocorrelation function. Stationarity.
LECTURES 2-3 : Stochastic Processes, Autocorrelation function. Stationarity. Important points of Lecture 1: A time series {X t } is a series of observations taken sequentially over time: x t is an observation
More information