Optimal control of linear systems with quadratic cost and imprecise input noise Alexander Erreygers

Transcription

1 Optimal control of linear systems with quadratic cost and imprecise input noise Alexander Erreygers Supervisor: Prof. dr. ir. Gert de Cooman Counsellors: dr. ir. Jasper De Bock, ir. Arthur Van Camp Master s dissertation submitted in order to obtain the academic degree of Master of Science in Electromechanical Engineering Department of Electrical Energy, Systems and Automation Chairman: Prof. dr. ir. Jan Melkebeek Faculty of Engineering and Architecture Academic year

2

3 Optimal control of linear systems with quadratic cost and imprecise input noise Alexander Erreygers Supervisor: Prof. dr. ir. Gert de Cooman Counsellors: dr. ir. Jasper De Bock, ir. Arthur Van Camp Master s dissertation submitted in order to obtain the academic degree of Master of Science in Electromechanical Engineering Department of Electrical Energy, Systems and Automation Chairman: Prof. dr. ir. Jan Melkebeek Faculty of Engineering and Architecture Academic year

4

5 First and foremost, I would like to thank my supervisor professor Gert de Cooman and my two counsellors Jasper De Bock and Arthur Van Camp. Expressing my gratitude for what you have done is not an easy task, but I will give it a try. When I first showed interest in doing my dissertation under your supervision, you instantly made me feel welcome. You created an extra dissertation proposal that was more my cup of tea, and I am very grateful for that. After introducing me to imprecise probabilities, you showed me the entrance to the optimal control maze. I stumbled upon several obstacles on my way through the maze; I would have been stuck in the maximality swamp or I would have fallen into the principle of optimality abyss if you guys would not have been there to lead the way. Thanks to your excellent guidance, I finally managed to find the exit of the maze after nine interesting months. I thank you for the many interesting discussions, your patience and your notes on the many drafts of my dissertation. Professor De Cooman, thanks for finding time for me in your already much too busy schedule. Jasper, thanks for making me think more like a mathematician and less like an engineer, and thanks for being so determined to solve the problems we encountered. Arthur, thanks for your help, your enthusiasm and all the food you offered me, especially the chokotoffs. Finally, I thank the three of you for creating the pleasant atmosphere in which I could work on my dissertation. I also thank Stavros and the other SYSTeMS members who made lunch breaks at Ally s so enjoyable. Mom and dad, thanks for supporting me in every possible way. You gave me the choice and opportunity to study what I wanted, and for that I will be forever grateful. I thank also the rest of my family, your warmth and support are much appreciated. Thank you Ginger for your love, thanks to my friends for their awesomeness and thanks to de Matabelen for letting me blow off steam in the weekend. Finally I thank Douglas Adams for his Hitchhikers Guide to the Galaxy series, Twinings for their excellent tea and Dell for making a laptop which can survive a tea spill. Permission for usage The author gives permission to make this master dissertation available for consultation and to copy parts of this master dissertation for personal use. In the case of any other use, the copyright terms have to be respected, in particular with regard to the obligation to state expressly the source when quoting results from this master dissertation. Alexander Erreygers 22nd May 2015

6

7 Optimal control of linear systems with quadratic cost and imprecise input noise by Alexander Erreygers Master s dissertation submitted in order to obtain the academic degree of Master of Science in Electromechanical Engineering Academic year Supervisor: prof. dr. ir. Gert De Cooman Counsellors: dr. ir. Jasper De Bock, ir. Arthur Van Camp SYSTeMS Research group Chairman: prof. dr. ir. D. Aeyels Department of Electrical Energy, Systems and Automation Chairman: prof. dr. ir. Jan Melkebeek Faculty of Engineering and Architecture Ghent University Abstract The problem of linear-quadratic optimal control of stochastic discrete-time linear systems is well known. This problem is usually solved using a precise uncertainty model and an independence assumption. The resulting optimal control is a combination of linear state feedback and recursively calculated noise feedforward. In this dissertation, we solve this problem using a more general imprecise uncertainty model a set of precise uncertainty models and a forward irrelevance assumption. Every element of the set of E-admissible control policies that is derived using these more general assumptions has the same combination of linear state feedback and possibly different recursively calculated noise feedforward. While calculating this feedforward is not tractable for a single precise model in the imprecise model, we show that for a scalar discrete-time linear system all the possible feedforward terms are elements of an interval. This interval has an upper and a lower bound that can be easily calculated recursively. Keywords Linear system, quadratic cost, optimal control, imprecise noise, forward irrelevance

8

9 Optimal control of linear systems with quadratic cost and imprecise input noise Alexander Erreygers Supervisors: prof. dr. ir. Gert de Cooman, dr. ir. Jasper De Bock, ir. Arthur Van Camp Abstract The stochastic linear-quadratic optimal control problem for discrete-time linear systems is a well known problem. This problem is usually solved using a precise uncertainty model with an independence assumption. We solve this problem using a more general imprecise uncertainty model and the less strict forward irrelevance assumption. Keywords Linear system, quadratic cost, optimal control, imprecise noise, forward irrelevance. I. Introduction THE optimal control of linear systems has been the subject of a large amount or research, see for instance [1]. When the system is disturbed by noise, we use probability theory to model our knowledge about this noise and to define an optimality criterion. Usually, a precise probability model a single joint probability density function is used. We use an imprecise probability model for the noise, which we define as a set of probability density functions. II. Optimal control of linear systems We consider a finite-state, discrete-time scalar linear system with a deterministic known initial state X 0 = x 0. We are interested in the evolution of the systems until the final time instant n, where n is a natural number. For all time instants k {0, 1,..., n}, the dynamics of the system is described by X k+1 = a k X k + b k u k + W k. 1 In this expression, a k and b k are real-valued parameters and the state X k and noise W k at time k are real-valued random variables. The control input u k at time k is also real-valued. If at all time instants k {0, 1,..., n}, a k = a and b k = b, then we call the system stationary. Our goal is to control the state x k of the system to zero in the most efficient way possible. Usually, the control input u k at time k is taken to be a function ψ k of the previous states x 0:k = x 0, x 1,..., x k. Such a function ψ k is called a feedback function, and we call a tuple of feedback functions ψ 0:n := ψ 0, ψ 1,..., ψ n a control policy. A very popular way to measure the performance of such a control policy ψ 0:n is the linear-quadratic cost functional η, see for instance [1]. This LQ linear-quadratic cost functional η is defined for all initial states x 0 R and all control policies ψ 0:n as η [ψ 0:n x 0 ] := n r k ψ k X 0:k 2 + q k+1 Xk+1. 2 k=0 In this expression, r k is a strictly positive real number and q k+1 is a non-negative real number. If the linear system is stationary, then the the coefficients r k and q k+1 are chosen to be the same at all time instants k. III. Model for the uncertainty of the noise We need to model the uncertainty on the noise W 0:n := W 0, W 1..., W n. Traditionally, a precise probability model a probability density function is used. The joint probability density function f 0:n defined on R n+1 models our beliefs about the noise W 0:n. If the joint probability density functions satisfies some technical conditions, then we can calculate the conditional probability density functions f k w 0:k 1 defined on R, which are defined for all k {0,..., n} and all w 0:k 1 := w 0, w 1,..., w k 1 R k. If k = 0, then w 0: 1 is not defined and we simply write f 0. For every joint probability density function f 0:n, we define the linear prevision of the gamble g a real-valued function on R n+1 as E 0:n g := + + gw 0:n f 0:n w 0:n dw 0 dw n, if this Riemann integral exists. For each of the conditional probability density functions f 0:k w 0:k 1, we define the conditional linear prevision of the gamble g on R as E k g w 0:k 1 := + gw k f k w k w 0:k 1 dw k, if this Riemann integral exists. Under some minor technical assumptions, the law of iterated expectation holds. This law states that for all k {0,..., n 1} and all w 0:k 1 R k the conditional linear prevision E k:n g w 0:k 1 of the gamble g on R n+1 k is given by E k Ek+1... E n g w 0:k 1, W k:n 1... w 0:k 1, W 0 w0:k 1, if these previsions exist. IV. The general case Initially, we use a single joint probability density function f 0:n. Under some relatively weak technical assumptions, there will be a unique control policy ˆψ 0:n that minimises the expected value E 0:n η [ψ 0:n x 0 ] of the cost. For all k {0,..., n} the optimal control policy is given by ψ k x 0:k := r k b k mk+1 a k x k + h k w0:k 1, 2 where the parameters r k, m k+1 are derived from the initial condition m n+1 := q n+1 and, for all k {0,..., n}, the

10 backwards recursive expressions r k := r k +b 2 k m k+1 1 and m k := q k + a 2 k r krm k+1. The term h k w0:k 1 is recursively calculated feedforward of the future noise, conditional on the noise history w 0:k 1. The feedforward is derived from the initial condition h n+1 w0:n := 0 for all w 0:n R n+1 and, for all k {0,..., n} and all w 0:k 1 R k, from the recursive expression h k w0:k 1 := E k mk+1 W k + r k+1 a k+1 r k+1 h k+1 w0:k 1 W k w0:k 1. 3 If the state realisations x 0:k are known, then we can calculate the noise realisations w 0:k 1 with Eq. 1. Therefore, the optimal control at time k is indeed a function of the states x 0:k. Unfortunately, calculating the actual feedforward without extra assumptions is intractable. In the following two sections we discuss two assumptions about the noise which simplify the calculations of the feedforward recursion. V. White noise A popular assumption in the literature is to assume that the noise terms at different time instants are independent. Noise terms W 0:n that satisfy this independence assumption are called white noise. Under this white noise assumption, the optimal control policy is again given by Eq. 2, where the parameters r k and m k+1 are derived from the same initial conditions and the same recursive expressions as in the general case. The benefit of the independence assumption is that all the conditional dependencies in the recursive expressions for the feedforward term h k can be neglected. The feedforward term h k is derived from the initial condition h n+1 := 0 and the recursive expression h k := m k+1 E k W k + r k+1 a k+1 r k+1 h k+1. 4 Stationary linear systems have an interesting property. If the expected value E k W k of W k is the same for all k {0..., n}, then it can be shown that for n, under some minor technical condition, the term m 0 converges to the limit value m and the feedforward term h 0 converges to the limit value h. VI. Forward irrelevant noise What is novel in our approach of the optimal control problem, is that we use an imprecise uncertainty model for the noise terms W 0:n. We model our beliefs about the noise terms W k using a set of precise models, i.e. using a set of joint probability density functions. We use the E- admissibility optimality criterion, one of the possible imprecise optimality criteria, see [3]. A control policy ψ 0:n is E-admissible if it is optimal for at least one of the precise models in the imprecise model. Instead of assuming that the noise terms W 0:n are independent, we assume that they are forward irrelevant. Forward irrelevance is an imprecise notion of independence, see [2]. Under this forward irrelevance assumption the corresponding set of optimal control policies is again characterised by Eq. 2. Unfortunately, calculating the feedforward terms h k is still not tractable. However, the forward irrelevance assumption allows us to find bounds for the feedforward terms of all the E-admissible control policies. We show that, under some minor technical conditions, the exact lower and upper bounds of this interval are h k = m k+1 E k W k + r k+1 a k+1 r k+1 h k+1, h k = m k+1 E Wk + r k+1 a k+1 r k+1 h k+1, with h n+1 = h n+1 := 0, and E k W k and E k W k the lower and upper expectations of W k. If a < 0, then h k+1 and h k+1 switch places. At first sight, these bounds might seem to follow trivially from Eq. 4, but this is not the case. The optimisation ranges over a forward irrelevant set of joint probability density functions, almost none of whose members corresponds to white noise. In this way, for any time k and state history x 0:k, we obtain an interval of optimal control inputs. Nevertheless, in a practical control situation a single control input has to be chosen. The most obvious or lazy choice is to apply the control input which, amongst the ones in the interval, has the lowest absolute value. In future work, we would like to investigate how this lazy choice performs in practice. Stationary linear systems again have an interesting property. We assume that the lower expectation E k W k is equal for all time instants k {0,..., n}, as is the upper expectation E k W k. As the recursive expressions are the same, for n the term m 0 will obviously converge to the same limit value m. We show that the lower bound h 0 and the upper bound h 0 of the feedfoward interval converge to the limit values h and h. VII. Conclusion Using the LQ cost functional we defined an optimality criterion for control policies. We first derived the optimal control policy using a precise model for the noise, and showed that it is a combination of linear state feedback and noise feedforward. Recursively calculating the feedforward of this policy turned out to be intractable. We then made two simplifying assumptions. The first assumption was the classical precise white noise assumption, which simplified the calculations for the feedforward a lot. Unfortunately the independence assumption is rather strict, and is often hard to justify. The second, more general assumption was the forward irrelevant noise assumption. This assumption results in intractable recursive expressions too, but we have shown that we can calculate lower and upper bounds for the E-admissible feedforward terms. References [1] Special Issue on the Linear-Quadratic-Gaussian Estimation and Control Problem, volume AC-16 of IEEE Transactions on Automatic Control. IEEE, New York, December [2] Gert de Cooman and Enrique Miranda. Forward irrelevance. Journal of Statistical Planning and Inference, 1392: , [3] Matthias CM Troffaes. Decision making under uncertainty using imprecise probabilities. International Journal of Approximate Reasoning, 451:17 29, 2007.

11 Contents 1 Introduction and preliminaries Introduction The structure of this dissertation Mathematical preliminaries Numbers Set theory Linear algebra Imprecise probabilities Introduction General terminology Random variable Possibility space Realisation Gamble Desirable gambles Lower previsions Coherent lower previsions Examples of coherent lower previsions Precise probabilities on finite possibility spaces Probability measures Probability mass functions Linear prevision of gambles on finite possibility spaces Precise probabilities on infinite possibility spaces Probability density functions Linear prevision of gambles on infinite possibility spaces Lower envelopes Lower envelopes for finite possibility spaces Lower envelopes for infinite possibility spaces Epistemic and forward irrelevance Extending stochastic independence Forward irrelevance vii

12 2.9 Decision making using probabilities Decisions and their consequences Decision making using precise probabilities Decision making using imprecise probabilities Why do we use imprecise probabilities? Linear Systems Introduction Systems theory Dynamical systems State space models Discrete-time linear systems Canonical form of a discrete-time linear system Controllability and observability Linearisation of non-linear systems Discrete-time control Stochastic discrete-time linear systems Discrete-time linear systems with stochastic disturbances State feedback Noise model Optimal control Introduction Deterministic optimal control Optimality of control inputs The deterministic LQ problem Stochastic optimal control Optimality of a control policy The linear-quadratic cost funtional Dynamic programming Solution of the stochastic LQ problem for general noise Single time step Two time steps Multiple time steps Calculating the feedforward White noise Forward irrelevant noise Scalar linear systems Introduction Scalar linear systems White noise Applying the optimal control Stationary linear systems and convergence of the recursion equations Forward irrelevant noise Bounds on the admissible feedforward terms

13 5.4.2 Stationary linear systems and convergence of the bounds of the admissible feedforward Control strategies Simulations Set up First simulation Second simulation Discussion Conclusion 75 Bibliography 77 MatLab code 79

14

15 List of figures 2.1 Example of the graphical representation Example of two lower previsions A black box system Convergence of m l and h l Convergence of m l and the bounds of the admissible feedforward interval First simulation - state and control input evolution Second simulation - state and control input evolution xi

16 List of symbols Symbols used only in Chapter 2. X, Y random variable X, Y possibility space of the random variable X or Y A, B event x, y realisation of the random variable X or Y g, h gamble L X set of gambles on the possibility space X K subset of the set of gambles I A indicator of the event A P g lower prevision of the gamble g P g upper prevision of the gamble g Pg linear prevision of the gamble g µ probability measure p X probability mass function for the random variable X p X,Y joint probability mass function of X and Y p X Y y probability mass function for the random variable X conditional on Y = y P g y linear prevision of the gamble g conditional on Y = y n, m natural numbers f X marginal probability density function for the random variable X f X,Y joint probability density function for the random variables X and Y f X y probability density function for the random variable X conditional on Y = y M set of dominating linear previsions D set of decisions S subset of the set of decisions D d decision Symbols used throughout Chapters 3 to 5. k 0 initial time of the noise model k current time l time index final time k 1 xii

17 X k n x X k x k x k:l u k U k n u y k Y k n y A k, B k W k W k w k w k0 :k state space at time k dimension of the state space state at time k as a random variable realisation of the state at time k state trajectory form time k until time l control input at time k control space at time k dimension of the control space output at time k output space at time k dimension of the output space system matrices at time k input noise at time k possibility space of W k realisation of the input noise at time k noise history until time k ψ k:k1 control policy from k until k 1 Ψ k0 :k 1 set of well-behaved control policies f k w k0 :k 1 probability density function of W k conditional on the noise history w k0 :k 1 P k:k1 g w k0 :k 1 linear prevision of the gamble g on W k:k1 conditional on the noise history w k0 :k 1 P precise noise model P imprecise noise model Q k credal set of marginal probability density functions of W k η LQ cost functional Q k, R k matrix coefficients of the LQ cost functional at time k loc-opt P local optimality operator associated with the precise noise model P opt P optimality operator associated with the precise noise model P ˆψ k0 :k 1 optimal control policy from k 0 until k 1 h k wk0 :k 1 feedforward at time k conditional on the noise history w k0 :k 1 h k feedforward at time k M k, N k, Rk matrices used in the definition of the optimal control policy a k, b k scalar system parameters at time k r k, b k scalar coefficients of the LQ cost functional at time k m k, n k, r k coefficients used in the definition of the optimal control policy m upper bound for m k m limit value for m k h limit value for h k h k, h k lower and upper bounds of the LQ-admissible feedforward interval at time k h, h limit values for h k, h k

18

19 Chapter1 Introduction and preliminaries 1.1 Introduction Control theory is a well-established research field. Since ages, mechanisms have been used to automatically control devices. The most well-known example of such a mechanism is the fly-ball governor of James Watt, which he used to control the speed of a steam engine. Correctly tuning this governor, however, was not an easy task. This tuning problem inspired James Maxwell to write On Governors in 1868, the first paper on the subject of mathematical control theory. He reduced the problem of tuning the governor to the mathematical problem of stability of differential equations [1]. Classical control theory uses transfer functions to model a dynamical system. However, since the late 1950s research in control theory started to focus more on the state space approach. One of the fields of the state space approach is the study of linear systems [1]. In the field of linear systems there are a couple of well-known problems, which are so well-known because their solutions have closed-form expressions. Examples of these well known results are the Kalman filter and the very closely related problem of stochastic linear-quadratic optimal control. Both of there results are treated in the Special Issue on the Linear-Quadratic-Gaussian Estimation and Control Problem of IEEE Transactions on Automatic Control. The linear-quadratic stochastic optimal control of discrete-time systems is the subject of this dissertation. The solution to this problem is a very well-known closed-form expression. However, this closed-form expression is the consequence of some very strict assumptions. It is obtained using a precise probability model and an independence assumption. Uncertainty cannot always be modelled using a precise probability model, and the independence assumption is even harder to justify. Other probability theories than precise probability theory have been proposed. One of these is the theory of imprecise probabilities, a behavioural probability theory which was popularised by Walley in [2]. It is a more general probability theory: the theory of precise probabilities is a special case of the theory of imprecise probabilities. Therefore, it is interesting to apply the methods of this more general imprecise probability theory to problems that have already been solved using precise probability theory. This is exactly what we will do for the problem of 1

20 2 Chapter 1: Introduction and preliminaries stochastic linear-quadratic optimal control of discrete-time linear systems. 1.2 The structure of this dissertation Hereafter this dissertation is subdivided into 5 chapters. Before starting with the main text, the reader is encouraged to read the next section, in which some mathematical preliminaries are introduced. Chapter 2 is a short introduction to imprecise probabilities. In the first sections we explain how this theory can be used to model the beliefs of someone about a certain random variable. We also introduce precise probabilities. We will see that precise probabilities are a special case of imprecise probabilities, and that precise probability models can be used to build imprecise probability models. The second to last section is an introduction to making decisions under uncertainty. Finally, we finish the chapter with a short motivation for using imprecise probabilities. In Chapter 3 we introduce linear systems, which are the main focus of this dissertation. We start by giving a general overview of linear systems theory, but quickly limit ourselves to discrete-time linear systems. We motivate the use of discrete-time linear systems, and introduce the concept of state feedback. We end this chapter with a model for discrete-time linear systems that are disturbed by input noise. We use the theories introduced in Chapters 2 and 3 to formulate the optimal control problem in Chapter 4. The introduction is general, but we quickly limit ourselves to the linear-quadratic optimal control problem. We derive the optimal control using a general noise model, and prove that it is a combination of state feedback and noise feedforward. We argue that we need extra simplifying assumptions on the noise model in order to make calculating this feedforward tractable. To round up this chapter, we mention two possible simplifying noise models, namely the white noise model and the forward irrelevant noise model. In Chapter 5 we limit ourselves to the study of scalar linear systems. We illustrate just how much the white noise model and forward irrelevant noise model simplify the problem. We also show some convergence properties for stationary linear systems. We finish this dissertation with Chapter 6, where the main ideas of the dissertation are summarised and where some ideas for further research are given. 1.3 Mathematical preliminaries In a final introductory section, we introduce some of the mathematical preliminaries necessary to understand this text. We do not aim to give a sound and complete mathematical development of the introduced concepts. Instead, we use this section to make the reader acquainted with the notation that we use throughout this dissertation Numbers We will use R to denote the set of real numbers and N to denote the set of natural numbers. We say that a real or natural number is strictly positive if it is strictly larger than zero, i.e. r R is strictly positive if r > 0 and n N is strictly positive if n > 0. A real number r R is called non-negative if r 0. The symbol R >0 denotes the subset of the real numbers that

21 1.3 Mathematical preliminaries 3 are strictly positive. Similarly, the symbol R 0 denotes the subset of the real numbers that are non-negative Set theory The following definitions are taken from [3]. Definition 1.1 The set that contains no elements is called the empty set. We denote the empty set with. Definition 1.2 Let A and B be two sets. If every element of A is also an element of B, then we say that the set B includes the set A and we write A B. Definition 1.3 Let A and B be two sets. The union of the set A and the set B is defined to be the set that contains all elements only in A, all elements only in B and all elements both in A and in B. We denote the union of A and B with A B. Definition 1.4 Let A and B be two sets. The intersection of the set A and the set B is defined to be the set that contains all elements both in A and in B. We denote the intersection of A and B with A B. Definition 1.5 Let A and B be two sets. If the intersection of A and B is the empty set, then we say that A and B are disjoint. Definition 1.6 Let A be a non-empty set. The largest subset of A that does not contain any elements of the set B is denoted by A \ B Linear algebra We only introduce the concepts of linear algebra which are necessary for this dissertation. The following is based on [4], whereto we also refer for a more general introduction. Notation In this dissertation we use lower-case letters to denote vectors. Let n be a natural number, and v be a vector in some n-dimensional vector space V. The vector v has n components with respect to some basis B of V, and we use v i to denote the i-th component of the vector v, i = 1, 2,..., n. In this dissertation the vector space V will always be the real n-dimensional vector space R n. The symbol 0 denotes the origin or the identity element for vector addition of the real n-dimensional vector space. We do not differentiate between the real number 0 and the origin of the real vector space, we assume that the meaning of the symbol 0 will be obvious from the context. We use bold upper-case letters to denote matrices. In this dissertation we only deal with real matrices. Let n and m be natural numbers, and M be a matrix in the set R n m. The matrix

22 4 Chapter 1: Introduction and preliminaries M has n rows and m columns, and we use the notation M i,j for the element of M of the i-th row and the j-th column, i = 1, 2,..., n and j = 1, 2,..., m. Some properties and definitions Definition 1.7 Let m and n be natural numbers, and let M be a real matrix of dimension n m. The transpose of the matrix M is denoted by M T and defined as the real matrix in R m n, which for all i {1, 2,..., m} and for all j {1, 2,..., n} is defined as M T i,j := M j,i. Definition 1.8 The matrix M is called square if it has the same number of rows as columns. Definition 1.9 Let n be a natural number and let M be a square matrix of dimension n. If M T = M, then we call M a symmetric matrix. A special square and symmetric matrix is the identity matrix, denoted by I. The diagonal elements of the identity matrix are 1, all the other elements are zero. For all i, j {1, 2,..., n} the i, j component of the identity matrix I of dimension n n is given by I i,j := { 0 if i j, 1 if i = j. The dimension of the identity matrix should be obvious from the context. Square symmetric matrices are often used to abbreviate the notation of quadratic forms. A quadratic form q is a homogeneous polynomial of degree 2. Let n be a natural number. The quadratic form q of n real-valued variables x 1, x 2,..., x n is given by q x 1, x 2,..., x n = n n a i,j x i x j. i=1 j=1 Now if we let x be the n-dimensional vector with components x i = x i and A be the square n-dimensional matrix with components A i,j := a i,j, then we can write q x 1, x 2,..., x n = x T Ax. Some square and symmetric matrices have a special quadratic form. Definition 1.10 Let n be a natural number. The square symmetric n-dimensional matrix M is called positive semi-definite if for all x R n it holds that We write M 0. x T Mx 0.

23 1.3 Mathematical preliminaries 5 Definition 1.11 Let n be a natural number. The square symmetric n-dimensional matrix M is called positive definite if it is positive semi-definite and if We write M > 0. x T Mx = 0 x = 0. Definition 1.12 Let n be a natural number, and let M be an n-dimensional square matrix. We say that the matrix M is invertible if there is a matrix M 1 R n n that satisfies M 1 M = I. If it exists, then this M 1 is unique and called the inverse of M. We mention the following theorem without proof. Theorem 1.13 [4] Let n be a natural number, and let M be an n-dimensional square symmetric matrix. If M is positive definite, then it is invertible.

24 Chapter2 Imprecise probabilities 2.1 Introduction Every thinking organism on our planet deals with uncertainty every day. We are not talking about uncertainty in the sense of doubting oneself, but about uncertainty in the sense of not being sure if you have to take your towel with you in the morning or not. If you believe the Hitchhiker s Guide to the Galaxy, you should always take your towel with you. Most people, however, do not like to carry their towel around with them all the time. They only like to carry their towel if they are actually going to need it. The problem is that you can never be entirely sure that you will not need your towel. This uncertainty is a problem, as it makes deciding whether or not to take your towel with you in the morning difficult. To deal with uncertainty, we should approach it in a rational and scientific way. Since long, people have been trying to model uncertainty using mathematics. The first applications of probability theory were games of chance. Nowadays, probability theory is used in science, engineering and economics to account for uncertainty and variability in processes [3]. We briefly introduce the theory of imprecise probabilities, which has been gaining momentum ever since Walley published [2] in We also show how the theory of precise probabilities follows from the theory of imprecise probabilities, and how we can use sets of precise probability models to define an imprecise probability model. The concepts and definitions in this chapter are taken from [2, 3, 5]. 2.2 General terminology In this section, we introduce the basic concepts and terminology that are used in probability theory. All concepts and terms are illustrated by means of the same example Random variable We use probability theory because we are uncertain about the actual value of some random variable X. A random variable can be thought of as the uncertain outcome of an experiment. In this dissertation a random variable will always be denoted by an upper-case character. 6

25 2.2 General terminology 7 Example Two boys, Andrey and Peter, are playing with a toy revolver that shoots small rubber balls. They decide to play a game of Russian Roulette. In this game, one boy puts a single rubber ball in the toy revolver, spins the cylinder and hands the revolver over to the other boy. The other boy then puts the muzzle of the revolver to his temple and pulls the trigger. The random variable X is the outcome when one of the boys pulls the trigger Possibility space We assume that although we are uncertain about the actual value of the random variable, we have information about which possible values the random variable can take. The non-empty set of possible values is called the possibility space X. Any subset A of a X is called an event. Example Because Peter is the younger of the two, he gets to start. He puts a single rubber ball in the magazine of the revolver, spins the cylinder and hands the revolver over to Andrey. When Andrey pulls the trigger, there are three possible outcomes: x 1 : the chamber that is aligned with the hammer contains the rubber ball and the rubber ball is fired, x 2 : the chamber that is aligned with the hammer is empty and nothing is fired, x 3 : the revolver jams. The possibility space of the random variable X is X = {x 1, x 2, x 3 }. Note that there are actually many more possible but very unlikely outcomes. For instance, the outcome that a humpback whale suddenly materializes and crushes Andrey is very unlikely. When modelling a problem, we always have to find a balance between realism and tractability. When we define a possibility space, we limit ourselves to possibilities in which we are interested in Realisation After the experiment is conducted we observe the actual value or the realisation x of the random variable X. This actual value x is an element of the possibility space X. In this dissertation the actual value of some random variable will always be denoted by the lower case version of the character used to denote the random variable. Example Andrey pulls the trigger. To Peter s great disappointment, Andrey is lucky and the hammer hits an empty chamber. The realisation of the random variable X is x Gamble A gamble is an uncertain reward g that depends on the realisation x of the random variable X. The reward g is expressed in a linear utility scale and can be negative. Examples of a linear utility scale are small amounts of money or tickets for a lottery for which every ticket has the same chance of winning [2].

26 8 Chapter 2: Imprecise probabilities Definition 2.1 A gamble g is a bounded function that maps the possibility space X to the real numbers: g : X R x gx. The set of all gambles L X is the set of all possible gambles on the possibility space X. Example Peter and Andrey make a deal. Peter will load the revolver, and Andrey will shoot it. When the revolver fires the rubber ball, Andrey gets 15. When the revolver does not fire the rubber ball however, Andrey has to give Peter 5. They agree that if the revolver jams, then no utility is exchanged. From the perspective of Andrey, the deal the two boys have just made is equal to the gamble g, defined for all x X as 15 if x = x 1, gx := 5 if x = x 2, 0 if x = x 3. A special type of gambles are the indicator functions. Let A be an event. An indicator function I A is a function I A : X {0, 1}, defined for all x X as I A x := { 1 if x A, 0 if x A. 2.3 Desirable gambles Imprecise probability theory is a behavioural probability theory. Uncertainty is modelled as the belief of some expert, from here on referred to as the subject. The subject is presented a gamble, which is an uncertain reward whose value depends on the outcome of some experiment. The fundamental idea behind imprecise probability theory is that the subject s knowledge about the experiment is represented by his tendency to accept or decline this gamble. If for instance the subject is sure that the gamble will result in a loss of utility, he will not be inclined to accept it. The subject s beliefs about the experiment can be modelled as follows. The subject could think of a gamble g L X as being desirable. This desirability is a result of his knowledge about the random variable X. Finding a gamble g desirable is equivalent to preferring the uncertain reward gx over zero. The set D of gambles that the subject prefers over zero is his set of desirable gambles. If the subject finds it desirable to buy the uncertain reward g for a price λ, then the gamble g λ should be an element of D. Definition 2.2 A set of desirable gambles D is called coherent if all g, h L X and all λ R >0 the four coherence axioms are satisfied: D1 g > 0 g D, accepting sure gain

27 2.4 Lower previsions 9 D2 g 0 g D, avoiding sure loss D3 g, h D g + h D, D4 g D λg D. Axioms D1 and D2 are a consequence of the rational behaviour of the subject. The subject will definitely find any gamble desirable where he is sure not to lose any utility and has the chance to gain utility. Likewise, a rational subject would not think desirable any gamble that is sure not to win him any utility. Axioms D3 and D4 are a consequence of the linearity of the utility scale. From axioms D3 and D4 it follows that D is closed under finite positive linear combinations. We define the posi operator for all sets of gambles K L X as { n } posik := λ i f i : n N >0, λ i R >0, f i K, 2.1 i=1 where N >0 is the set of strictly positive natural numbers and R >0 is the set of strictly positive real numbers. For any coherent D it holds that posid = D; any coherent set of desirable gambles is a convex cone. The set which includes all coherent sets of desirable gambles is denoted by D. If for D 1, D 2 D it holds that D 1 D 2, then we say that D 1 is more conservative than D 2. The relation defines a partial order relation on D. 2.4 Lower previsions The theory of sets of desirable gambles leads to the associated theory of lower previsions. Two transactions with the reward g are considered: 1. buying the uncertain reward g for a price µ, which is equivalent to the gamble g µ; 2. selling the uncertain reward g for a price λ, which is equivalent to the gamble λ g. Definition 2.3 The lower prevision P g of a gamble g is defined as P g := sup {α R: g α D}. The lower prevision P of a gamble g is the subject s supremum buying price, or in other words it is the smallest α such that the subject desires to buy g for all prices µ < α. Definition 2.4 The upper prevision P g of a gamble g is P g := inf {β R: β g D}. The upper prevision P of a gamble g is the subject s infimum selling price, or in other words it is the smallest β such that the subject desires to sell g for all prices λ > β. Note that nothing is said about the buying price µ = P g or the selling price λ = P g.

28 10 Chapter 2: Imprecise probabilities It is easy to see that the subject s supremum buying price and infimum selling price need not be the same. In general, his supremum buying price will be lower than his infimum selling price Coherent lower previsions Definition 2.5 A lower prevision P is called coherent if for all g, h L X and all λ R >0 the three coherence axioms are satisfied: P1 P g infg, P2 P g + h P g + P h, P3 P λg = λp g. super-additivity positive homogeneity It is argued in [5] that if the set of desirable gambles D is coherent, then the lower prevision P associated with this set is coherent. When talking about lower previsions, we usually do not rely on the subject s set of desirable gambles D. Instead, we assume that for any gamble g in some subset K L X the subject has assessed a lower prevision P g. This way, we have defined a functional P: K R, which is called a lower prevision with domain K. Lower and upper previsions are linked via conjugacy. If P is a lower prevision with domain K, then we can define the conjugate upper prevision with domain K = {f : f K } for all g K as P g := P g. We can restrict the domain of a coherent lower prevision P to events using indicator functions. For any event A X we write P A := P I A ; we call this lower prevision the lower probability of the event A Examples of coherent lower previsions Two special cases of coherent lower previsions are considered in this section. Example In this section we will, for the sake of simplicity, assume that the possibility space X consists only of x 1 and x 2. We will represent the desirable gambles and lower previsions graphically in the plane, in a similar manner as [5]. The realisation x 1 is the horizontal axis, the realisation x 2 the vertical axis. A gamble g is represented as a point gx 1, gx 2. An example of this graphical representation can be seen in Fig In Fig. 2.1 the set of gambles playing a role in D1 is shaded light grey, while the set of gambles that take a part in D2 is shaded dark grey. In the other examples, we will shade the set of desirable gambles light grey.

29 2.4 Lower previsions 11 x 2 g x 1-1 Figure 2.1: Example of the graphical representation Vacuous lower previsions Assume that for all gambles g L X the lower prevision is given by P g = inf g and the upper prevision is given by P g = sup g. It is obvious that these previsions are coherent. We call them vacuous previsions because they result in maximal imprecision P g P g = sup g inf g for any gamble g. Vacuous upper and lower probabilities are the restriction of vacuous upper and lower previsions to indicator functions of events. As Walley argued in [2], vacuous lower and upper probabilities model complete ignorance about a possibility space X better than the typical Bayesian uniform probability distribution. Example The vacuous lower prevision corresponds to a complete lack of knowledge, which means that the set of desirable gambles consist only of the gambles g 0 that are not zero everywhere. This can be seen in Fig. 2.2a too. What is also depicted in Fig. 2.2a is the lower prevision of a gamble x 2 x 2 g 1 P g p P g p g x x a Vacuous lower prevision b Linear prevision Figure 2.2: Example of two lower previsions

30 12 Chapter 2: Imprecise probabilities g. A way to graphically calculate the supremum of alpha such that g α is still desirable is to draw a line starting at g with slope 1. The point p where the slope and the boundary of the convex cone of desirable gambles intersect is important. If the intersection is to the left of the gamble, then P g > 0. Conversely, if the intersection is to the right of the gamble, then P g < 0. The supremum of α for which g α D can be found by measuring the length gx 1 p x1 or gx 2 p x2, which is indicated in Fig. 2.2a as P g. Linear previsions The coherent lower prevision P with domain K = K L X is called self-conjugate if for all g K it holds that P g = P g. As there is no difference between the lower prevision and the upper prevision, we will simplify the notation and write P g instead of P g or P g. For this self-conjugate lower prevision P, the coherence axioms of lower previsions simplify for all g, h K and all λ R to LP1 infg P g sup g, LP2 P g + h = P g + P h, LP3 P λg = λp g. This self-conjugate lower prevision is also called a linear prevision because the super-additivity is strengthened to normal additivity. The set of all linear previsions defined on L X is denoted by P L X. When the lower prevision is equal to its conjugate upper prevision, the linear prevision P g is the fair price for the gamble g. The term fair price was used by de Finetti to refer to his precision axiom [6]. He also introduced the term prevision to indicate the fair price of a gamble. The fact that in imprecise probabilities the term prevision is used instead of expectation marks the influence of de Finetti s ideas on the study of imprecise probabilities. Example The revolver has six chambers, which is why Andrey assesses the likelihood of x 2 to be 5 times higher than the likelihood of x 1. If the prices are fair, a gamble that results in a non-zero reward λ for x 2 and a zero reward for x 1 will cost more than a gamble which results in the same non-zero reward λ for x 1 and a zero reward for x 2. A graphical representation of a linear prevision is depicted in Fig. 2.2b. The specific form of the set of desirable gambles is a consequence of the assessment of Andrey that x 2 is five times more likely than x Precise probabilities on finite possibility spaces The linear previsions that were introduced in the previous section are actually just the ones that are defined using precise probabilities. We will see in Section 2.7 that we can also define coherent lower previsions using sets of linear previsions. For this reason we will first make ourselves more acquainted with precise probabilities before we continue with imprecise probabilities. The following section is largely based on [3]. We recall that we use P A to denote the probability that the event A occurs, or in other words the probability that the realisation of the random variable is an element of A.

31 2.5 Precise probabilities on finite possibility spaces 13 Definition 2.6 A random variable X with finite possibility space X is called a discrete random variable Probability measures Definition 2.7 A finitely additive probability measure µ on a possibility space X is a function that assigns a real number called the probability to any event A X and satisfies the following three axioms for all events A, B X : PM1 µ A 0, PM2 µ X = 1, PM3 A B = = µ A B = µ A + µ B. positivity unit measure additivity If our beliefs about the random variable X are modelled by a finitely additive probability measure µ, then for any event A X we write P A = µ A. Property 2.8 If µ is a finitely additive probability measure defined on the possibility space X, then µ = 0. Proof. From the definition of the empty set we know that X = X and X =. PM3 implies that µ X = µ X + = µ X + µ. From PM2 it follows that µ = 0. Property 2.9 Let µ be a finitely additive probability measure on the possibility space X, then for any A X it holds that 0 µ A 1. Proof. Let A be an arbitrary subset of X. By PM1 we have that 0 µ A. Let B be the event defined as B := X \ A. It is obvious that A B = and A B = X. From PM2 it follows that µ X = 1. Using PM3 gives 1 = µ X = µ A + µ B. From PM1 we know that µ B 0, which implies µ A 1. Why the explicitly state finitely additive probability measure? For any finite-length tuple A 1, A 2,..., A n of pairwise disjoint events, we can prove by induction of PM3 that n n µ A i = µ A i. Finitely additive probability measures are studied more in detail in [7]. i=1 Theorem 2.10 If P is a coherent linear prevision that is restricted to the set of indicator functions of all events, then P is also a probability measure. i=1