Introduction to Stochastic Optimization Part 4: Multi-stage decision

Transcription

1 Introduction to Stochastic Optimization Part 4: Multi-stage decision problems April 23, 29

2 The problem ξ = (ξ,..., ξ T ) a multivariate time series process (e.g. future interest rates, future asset prices, future demands, etc.) At times t =, t =,..., t = T, we make decisions x,..., x T.

3 Example: dynamic portfolio management value : mu =.9597 risk = : mu = risk = time stage mu Efficient frontier multirisk dynport risk 2: mu =.9945 risk = : mu = 2.99 risk = : mu = risk = : mu = risk = An efficient frontier using the (negative) multiperiod AV@R as risk functional

4 Scenario processes Instead of a single scenario variable ξ let us now consider a scenario process ξ = (ξ,..., ξ T ). A decision model is called multi-period, if the scenario process has more than one period. and it is called multi-stage, if the decisions are to be made at different times, say at times,, 2,... period : period 2: period 3: observation of observation of observation of the r.v. ξ the r.v. ξ 2 the r.v. ξ 3 stage : stage : stage 2: stage 3: t = t = t = 2 t = 3 decision decision decision decision x x x 2 x 3

5 Tree processes and filtrations A stochastic process (ν t ) is called a tree process, if the sigma-algebra generated by ν t and by (ν, ν 2,..., ν t ) coincide σ(ν t ) = σ(ν,..., ν t ). Let F t = σ(ν t ). Then F = (F,..., F t ) is a filtration, i.e. F t F t+. Conversely, every filtration is generated by a tree process. We use the notation ξ t F t if ξ t is F t measurable for all t, and ξ F if (ξ t ) is adapted to the filtration F, i.e. ξ t F t for all t.

6 The scenario process (ξ t ) is adapted to F if and only if the ξ t s are functions of ν t, i.e. ξ t = f t (ν t ). Decisions must be made in the indicated order: First choose x, then observe ξ, then choose x and so on. i.e. also the decision sequence (x t ) must be adapted to the filtration F. The requirement x F is called the non-anticipativity constraint. Information is a resource. The value of perfect information can be interpreted as shadow costs for the lack of information.

7 Tree processes on finite probability spaces are trees ω ; P{ω } =.2 5 ω 2 ; P{ω 2 } =.3 6 ω 3 ; P{ω 3 } =.3 7 ω 4 ; P{ω 4 } =.8 8 ω 5 ; P{ω 5 } =.4 9 ω 6 ; P{ω 6 } =.8 ν ν ν 2 For a given tree denote by n the predecessor and by n+ the set of successors for the node n.

8 The planning horizon is T, that is the last decision to be taken is x T. The success (cash-flow, income, change of wealth) Y t within period t is a function H t of all observations and decisions before stage t that is Y t = H t (x, ξ,..., x t, ξ t ). The objective of multi-stage financial optimization problems is to maximize the acceptability of the whole operation under the non-anticipativity constraints and possibly some additional operating constraints. Maximize in x = (x, x,..., x T ) : A[Y,..., Y T ; F,..., F T ] where Y t = H t (x, ξ,..., x t, ξ t ) subject to the non-anticipativity constraints x F and possible some more operational constraints ()

9 State-space models Sometimes we may identify states z t such as wealth, actual portfolio, etc. state z decision x r.v. ξ state z decision t = t = Figure: The dynamics of a state-space decision model x r.v. ξ 2 The multi-stage problem in state-space formulation reads Maximize in x : A[H (z ), H 2 (z 2 ),..., H T (z T ); F] under the system dynamics z t = g t (z t, x t, ξ t ) t =,..., T with initial condition z subject to constraints x t X (z t ) and the nonanticipativity constraints x F.

10 Multi-period acceptability and risk functionals Let Y,..., Y T be discounted cash-flow ( success ) process adapted to (F t ). An acceptability functional A(Y,..., Y T, F,..., F T ) is a real valued functional defined on a space of adapted processes (Y t ) F Information monotonicity: If F t F t, for all t, then A(Y,..., Y T ; F,..., F T ) A(Y,..., Y T ; F,..., F T ). Multi-period translation equivariance: If c t F t, then A(Y,..., Y t + c t,..., Y T, F,..., F T ) = E(c t ) +A(Y,..., Y T, F,..., F T ). Concavity. (Y,..., Y T ) A(Y,..., Y T ; F,..., F T ) is concave. Monotonicity: If Y () t Y (2) t, a.s. t =,..., T, then A(Y (),..., Y () T, F,..., F T ) A(Y (2),..., Y (2) T, F,..., F T ).

11 Examples for multi-period acceptability functionals The multi-period conditional distortion functional: A(Y,..., Y T ; F) = T w t E[A Jt (Y t F t )] t= Special case: the multi-period AV@R: AV@R β,w (Y,..., Y T ; F) = T w t E[AV@R βt (Y t F t )]. Here (w t ) are some weights measuring the relative importance of successes at different times. t=

12 Distances for stochastic processes If (Ξ, d ) and (Ξ 2, d 2 ) are Polish spaces then so is the Cartesian product (Ξ Ξ 2 ) with metric d 2 ((u, u 2 ), (v, v 2 )) = d (u, v ) + d 2 (u 2, v 2 ). Consider some metric d on R m, which makes it Polish (it needs not to be the Euclidean one). Then we define the following spaces Ξ = (R m, d) Ξ 2 = (R m P (Ξ, d), d 2 ) = (R m P (R m, d), d 2 ) Ξ 3 = (R m P (Ξ 2, d), d 2 ) = (R m P (R m P (R m, d), d 2 ), d 2 ). Ξ T = (R m P (Ξ T, d), d 2 ) All spaces Ξ,..., Ξ T are Polish spaces and they may carry probability distributions.

13 Nested distributions Definition. A Borel probability distribution P with finite first moment on Ξ T is called a nested distribution of depth T. For any nested distribution P, there is an embedded multivariate distribution P. The projection from the nested distribution to the embedded distribution is not injective. Notation for discrete distributions: probabilities: values: [ ] [ ] [ Left: A valid distribution. Middle: the same distribution. Right: Not a valid distribution ]

14 Examples for nested distributions [ 3. ] [ 3. ] [ ] The embedded multivariate, but non-nested distribution of the scenario process can be gotten from it:

15 Minimal filtrations.5.5 [ ] [ ].. [ ] Left: Not a valid nested distribution. Right: A valid one This fact leads to the concept of minimal filtrations.

16 Example This tree process is already minimal Left: this tree process is not minimal, Right: its minimal reduction.

17 Main Results Theorem. Under a mild condition on the basic problem (compound convexity of R), we may w.l.o.g. reduce the problem to solutions, which are measurable w.r.t. the minimal filtration. We define the nested distance between two nested distributions P and P as the Kantorovich distance of these distributions on the metric space Ξ T. Theorem. Let P, P be nested distributions and P, P be the pertaining multi-period distributions. Then d KA (P, P) d KA (P, P). Theorem. If, for the objective R[H(x, ξ,..., x T, ξ T )], R and H are Lipschitz, then the approximation error satisfies e(opt, Õpt) C d KA(P, P).

18 Example for the nested distance Let and P = N (( ) (, 2 )). P = [.29 ] [. ] [ ]

19 The nested distance is d(p, P) =.82. The distance of the multiperiod distributions is d(p, P) =.68.

20 The nested distance is d(p, P) =.2. The distance of the multiperiod distributions is d(p, P) =.67.

21 he distance algorithm Let ν () and ν (2) be two tree processes and let the scenario processes (the node values) be ξ () t = f () t (ν () t ) resp. ξ (2) t = f (2) t (ν (2) t ). The distance between the two nested distributions is defined in a backward recursion way through a distance d t between the node of process and process 2 at each stage t. Let N () t resp. N (2) t be the node set at stage t of the tree processes ν () resp. ν (2). We define the distances d t on N () t N (2) t for each t. The final value of the distance is the distance between the roots of the two processes. To calculate the distance, we have to solve T () t= #(N t ) #(N (2) t ) linear optimization problems, where #(N (i) t ) is the number of nodes of process i at stage t.

22 End of Recursion. For the last stage T, let for (u, v) N () T N (2) T d T (u, v) := ρ(f () T where ρ is any metric on R m. (2) (u), f T (v)), Backward Recursion Step. Suppose that the distances d t+ have been defined on N () t+ N (2) t+. Then for each pair (u, v) N () t N (2) t, the conditional distributions of ν () t+ ν() t = u resp. ν (2) t+ ν(2) t = v are well defined. Let now d t(u, v) := ρ(f () t (u), f (2) t (u))+d(ν () t+ ν() t = u, ν (2) t+ ν(2) t = v) where the distances between ν () d t+. t+ and ν(2) t+ are measured by Final Result. For t =, the node sets N () resp. N (2) are singletons and contain only the root. The distance d between these roots is the nested distance between the trees.

23 Further examples tree tree 2 tree 3 d(p (), P (2) ) = 3.9; d(p (), P (2) ) = 3.48 d(p (), P (3) ) = 2.52; d(p (), P (3) ) =.77 d(p (2), P (3) ) = 3.79; d(p (2), P (3) ) = 3.44

24 The value of information Let ν t be a tree process and F t = σ(ν t ) = σ(ν,..., ν t ) be the σ-algebra generated by ν t. We will work with the standard filtration resp. the clairvoyant s filtration F = (F,..., F T ) (2) F T = (F T,..., F T ). (3) The standard tree (left) represents the filtration F. The clairvoyant expansion (right) represents the clairvoyants filtration F T.

25 With the short notation H(x, ξ) for H(x, ξ, x,..., x T, ξ T ) we may reformulate the multi-stage decision problem as The clairvoyant s problem is A := max{a[h(x, ξ)] : x F}. (4) C := max{a[h(x, ξ)] : x F T }, (5) where F T is the clairvoyants filtration (3). Notice that the condition x F T is in fact no restriction at all. If the functional A is pointwise monotonic, one may interchange the order of the maximization and the application of the functional in the clairvoyant s problem, i.e. max{a[h(x, ξ)] : x F T } = A[max{H(x, ξ) : x F T }]. (6)

26 Let H denote the inner function in (6), i.e. H(ω,..., ω T ) = max{h(x, ω, x,..., x T, ω T ) : x t X }. (7) The clairvoyant knows this function and gets the objective value It is evident that C = A[ H(ξ,..., ξ T )]. C A, since the feasible set of (4) is contained in the feasible set of (5). We may call the difference the value of perfect information D = C A. D is a measure of multi-period deviation risk.

27 The conditional problem For any possible value v of the tree process ν t, denote by P {νt=v} the conditional probability conditioned on the event {ν t = v}. Under P {νt=v} the variables ν,..., ν t sit on the predecessors of v, i.e. their distribution are concentrated on singletons. The variables ν t+,..., ν T sit only on those nodes, which are successors of the node v. In particular, conditioning on a terminal node v leads to a deterministic process, the predecessor path of v. The same decision problem, which is solved on the original tree, can also be solved on all conditional subtrees. By the notation A P [ ] we express the fact that the probability measure P governs the tree process. Define now A t (v) = max{a P {νt =v}[h(x, ξ)] : x F} C t (v) = max{a P {νt =v}[ H(x, ξ)] : x F T } where H is given by (7). By this construction, (A t ) and (C t ) are processes which depend on the tree process, i.e. are adapted to F.

28 The value-of-perfect-information process Define D t = C t A t as the value-of-perfect-information process. It is a nonnegative process, which describes the evolution of risk in time. A is compound convex, if Lemma. A(Y ) E[A(Y F)]. (i) If A is compound convex, then the clairvoyant process C t is a submartingale. (ii) If A is compound linear, then C t is a martingale. (iii) If A is compound convex, then the optimality process A t is a submartingale. (iv) If A is compound linear (in particular if A is the expectation), then the value-of-information process D t is a supermartingale.