Measure-valued derivatives and applications

Transcription

1 Stochastic Models and Control Bad Herrenalb, April 2011

2 The basic problem Our basic problem is to solve where min E[H(X θ)] θ Θ X θ ( ) is a Markovian process, with a transition law which depends on a parameter θ the functional H(X θ ) may be H(X θ ) = h(x θ (T )) H(Xθ ) = h(x θ ( )) H(Xθ ) = T 0 h(x θ(t)) dt H(X θ ) = τ 0 h(x θ(t)) dt, where τ is some stopping time.

3 Examples Markov Systems (queueing, service, manufacturing) Suppose that θ denotes the parameters of a Markov System (queueing, inventory, renewal). Let X θ (t) be the state of the system at time t and X θ ( ) the steady state (if exists). Then E[h(Xθ (T ))] is the performance of the system at time T T 0 E[h(X θ(t))] dt is the expected integrated transient behavior E[h(X θ ( ))] is the expected stationary behavior. Finance Let X θ (t) describe the evolution of an underlying asset, for a parameter vector θ. Let h be the payoff function of a contingent contract. Then E[h(X θ (T ))] is the value of the contingent contract (European type). If τ is a stopping time, then E[h(X θ (τ))] is the value of the American type contingent contract. We want to estimate the sensitivity of the price w.r.t. the parameter θ (the Greeks).

4 Finite differences: The Kiefer-Wolfowitz procedure θ n+1 = θ n + a n h(x θn +c n ) h(x θn c n ) 2c n. Small c n small bias and large variance Large c n large bias and small variance Differentiability of h is required. 2 c n Left: Differences c n 0, Right: No convergence c n 0

5 The problem We are interested in finding the value of and - more generally - θ E[h(X θ)] k θ k E[h(X θ)], as well as a way to estimate it based on sampling. We formulate the problem in terms of the distribution µ θ of X θ.

6 The two paradigms for derivatives Let F θ (x) be the distribution function of µ θ θ F θ ( ) measure derivatives θ F 1 θ ( ) pathwise derivatives measure-valued derivatives finite differences (FD) infinitesimal perturb. (IPA) score-function method Malliavin calculus (part. int.)

7 Measure valued derivatives-mvd Let (R, d) be a metric space. To the family of Borel probabilities on (R, d), we associate a dual function space F such as the space of all bounded, continuous functions the space of all continuous functions h, such that h(u) K 1 + K 2 d p (u, u 0 ) Definition. The family of probability measures (µ θ ) θ Θ R on R is weakly differentiable w.r.t. the dual space F, if there is a finite signed measure µ θ such that for all h F [ 1 s h(w) dµ θ+s (w) ] h(w) dµ θ (w) h(w) dµ θ (w) as s 0. (Heidergott, Vasquez-Abad, Leahu, Xi-Ren Cao, G.P.,...)

8 Decomposing the weak derivative Any finite signed measure may be decomposed into its positive and negative part (Jordan decomposition). Since 1dµ θ = 1, we have that 1dµ θ = 0, i.e. the positive and the negative part have the same mass. Thus we may decompose the derivative object µ θ µ θ = c θ( µ + θ µ θ ) where µ + θ and µ θ are probability measures. The representation as a multiple of the difference of two probability measures µ θ = c(µ 1 µ 2 ) is not unique, however the constant c is minimal if the two parts µ 1 and µ 2 are orthogonal, i.e. if the decomposition is the Jordan decomposition.

9 Any triplet (c θ, µ + θ, µ θ ), such that for h F [ 1 s h(w) dµ θ+s (w) ] h(w) dµ θ (w) [ c θ h(w) d µ + θ (w) h(w) d µ θ (w) ], for s 0, is called a measure valued derivative triplet.

10 Gamma, Normal, and related distributions The Gamma(a, b) distribution has density 1 b a Γ(a) x a 1 exp( x/b) If X Gamma(1/2, 2σ 2 ) (i.e. χ 2 (1)), then X is distributed according to the positive part of a N(0, σ 2 ) distribution. If X Gamma(1, 2σ 2 ), (i.e. χ 2 (2)), then X is distributed according to a Raleigh distribution, which is a Weibull distribution with exponent 2. If X Gamma(3/2, 2σ 2 ), (i.e. χ 2 (3)), then X is distributed according to a Maxwell distribution.

11 Example: Gradient w.r.t. the variance parameter The family N(0, θ 2 ) The derivative w.r.t. θ as a signed measure

12 The Jordan Hahn decomposition of the signed measure representing the derivative The decomposition of the derivative in a Maxwell part and a normal part

13 Examples for measure valued derivatives Distribution µ θ Constant Positive part of Negative part of (θ varies) c θ the derivative: µ + θ the derivative: µ θ Poisson(θ) 1 Poisson(θ) + 1 Poisson(θ) Normal(θ, σ 2 ) 1/σ 2π θ + Raleigh( 1 ) θ - Raleigh( 1 ) 2σ 2 2σ 2 Normal(m, θ 2 ) 1/θ ds-maxwell(m, θ 2 ) Normal(m, θ 2 ) Exponential(θ) 1/θ Exponential(θ) θ 1 Erlang(2) Gamma(a, θ) a/θ Gamma(a, θ) Gamma(a + 1, θ) Weibull(α, θ) 1/θ Weibull(α, θ) [Gamma(2, θ)] 1/α

14 Higher derivatives The Normal family N(θ, σ 2 ) has first derivative 1 [ σ 2π, θ + Raleigh( 1 1 ), θ Raleigh( 2σ2 2σ 2 )] and second derivative [ 1 σ 2, ds-maxwell(θ, σ2 ), Normal(θ, σ 2 )].

15 Use of weak derivatives in sensitivity estimation If Ẋ + θ resp. Ẋ θ are distributed according to µ + θ resp. Ẋ θ, then c θ [h(ẋ + θ ) h(ẋ θ )] is a consistent estimate of θ E[h(X θ)]. (Academic) Example. Let H(θ) = E[cos(X θ )], where X θ is a Normal(0,θ 2 ) variable. Then where 1/θ[cos(Ẋ + θ ) cos(ẋ θ )] Ẋ + θ double-sided-maxwell(0, θ2 ) and Ẋ θ Normal(0, θ2 ) is an unbiased estimate for θ H(θ). Notice that no infinitesimal quantities appear and that we do not have to know the derivative of cos.

16 Coupling over a distance A coupling of two probability measures µ 1 and µ 2 w.r.t. h is a probability measure µ on R R with given marginals µ i, which minimizes the expectation of the criterion function d(u, v): Minimize d(u, v) µ(du, dv) subject to proj 1 µ = µ 1, proj 2 µ = µ 2, µ is a probability on R R The solution may not be unique. We denote the solution (or the set of solutions) by µ 1 c d µ 2 and call it µ 1 and µ 2 coupled over d.

17 Example Distribution µ θ Constant Positive part of Negative part of (θ varies) c θ the derivative: µ + θ the derivative: µ θ Poisson(θ) 1 Poisson(θ) + 1 Poisson(θ) Coupling over the Euclidean distance d(u, v) = u v leads to taking Ẋ + = X θ + 1, Ẋ = X θ, i.e.: For any integrable (summable) cost function h and any Poisson variable X θ Poisson(θ) we have with very low variance. θ E[h(X θ)] = E[h(X θ + 1)] E[h(X θ )]

18 The score function method If X θ has density f θ (x), then under appropriate conditions [ ] θ E[h(X θ)] = E h(x θ )ḟθ(x θ ) f θ (X θ ) where is the score function. ḟ θ (x) f θ (X θ ) = θ log f θ(x) The score function methods requires stronger assumptions than the MVD-method When using an appropriate coupling, the MVD-method leads to smaller variances. In a stochastic process situation, the score function martingale has typically large variance.

19 A comparison Estimation of sensitivity of θ E[ X θ ], where X θ Exponential(θ). Then the variances are Numerical differences Score function method 2.81 MVD with coupling 0.022

20 Extension to Markov Chains Let (P θ ) θ Θ be a family of Markov transitions on the metric state space (R, r). Definition. A (regular) finite signed transition operator T(w, A) is a mapping R B R with the property w T(w, A) is measurable for each Borel set A, A T(w, A) is a finite signed measure for each w. We introduce the following notations 1lµ is the transition T(w, A) = µ(a) µt is the measure (µt)(a) = T(w, A) dµ(w). Th is the function (Th)(u) = h(w)t(u, dw).

21 Definition. The Markov transition P θ (, ) is called (uniformly) weakly differentiable, if there is a signed transition P θ such that for all continuous functions h satisfying h(u) K(1 + u p ) for all u and some constant K and every point mass δ w (i.e. the probability distribution concentrated on the point w) as s 0, uniformly in w. 1 s δ w P θ+s h δ w P θ h s δ w P θ h 0

22 Every finite signed transition may be decomposed as T(w, A) = c(w)[p 1 (w, A) P 2 (w, A)], where P 1 and P 2 are regular Markov transitions. Again, we may select convenient decompositions: We choose two Markov transitions Ṗ + θ and Ṗ θ and a measurable function c θ(w) such that P θ (w, A) = c θ(w)[ṗ + θ (w, A) Ṗ θ (w, A)].

23 The Leibniz formula and its sampling counterpart (P n θ ) = n i=1 P i 1 θ P θ Pn i θ. Estimation of γ(p n θ ) h: 1. Sample a random uniform time τ in {1,..., n}. 2. Sample X θ (0) from the starting distribution γ. 3. Sample τ 1 steps with transition P θ, giving X θ (1),..., X θ (τ 1) 4. Sample one transition step from X θ (τ 1) with transition Ṗ + θ and one with transition Ṗ θ, giving Ẋ + θ (τ) resp. Ẍ θ (τ). 5. Continue the processes Ẋ + θ (t) resp. Ẋ θ (t), t = τ + 1,..., n using transition P θ and a coupling technique. 6. The final estimate is nc(x θ (τ 1))[h(Ẋ + θ (t)) h(ẋ θ (t))].

24 Ṗ + θ P γ θ P θ P θ Ṗ θ P θ P θ P θ P θ X θ + (t) X θ (t) X θ (t) coupling 0 τ T Alternatively, instead of sampling the stopping time τ, one may take the sum over all i without further sampling. We call this method exact measure valued derivation.

25 Gradients of stationary distributions Let π θ P θ = π θ for all θ be the (unique) stationary distribution of P θ. Then θ π θ is weakly differentiable and π θ = π θ P θ S θ where S θ is the inverse Poisson operator, satisfying S θ (I P θ + 1l π θ ) = I, with I being the identity operator. The von Neumann series for S θ is S θ = (P m θ 1l π θ). m=0

26 Estimation of θ π θh 1. Start with an arbitrary starting distribution 2. Do m steps with transition P θ to get X (m) θ 3. Make one transition with Ṗ + θ and one transition with Ṗ θ to get Ẋ + θ (0) resp. Ẋ θ (0) 4. Starting with Ẋ + θ (0) resp. Ẋ θ (0) do n steps with transition P θ to get sample Ẋ + θ (n) resp. Ẋ θ (n). These two processes should be coupled. 5. The final estimate is c(x (m) θ )[h(ẋ + θ (n)) h(ẋ θ (n))].

27 Higher derivatives: A simple observation Consider two Markov transition operators P and Q and set P θ = (1 θ)p + θq. Let π θ be the stationary distribution of P θ, i.e. π θ P θ = π θ. Introduce the deviation matrix D = (P 1l π 0 ) j. Then - at least for small θ - j=0 π θ = π 0 θ j [(Q P)D] j j=0 and therefore the expected costs for a cost function h (not depending on θ) are E πθ [h(x ( ))] = π θ, h = π 0 θ j [(Q P)D] j h. j=0

28 Example: the MM1 queue λ x One queue with arrival intensity λ One server with service intensity θ, which is the decision variable Intensity matrix: λ λ θ (λ + θ) λ θ (λ + θ) λ Costs: Probability that more than 3 customers are in the waiting queue.

29 Taylor expansion up to first order Left: The stationary distribution π θ. Right: The probability that more than 3 customers wait Taylor expansion up to fifth order

30 Lévy Processes and the estimation of Greeks X (t) is a Lévy Process, if it has independent, stationary increments. The pertaining price process is S(t) = S 0 exp(x (t)). We typically make an Esscher transform to find a measure Q under which exp( rt)s(t) is a martingale. We consider especially the following Lévy processes The Brownian motion The Poisson process The Compound Poisson process The Gamma process The Variance Gamma process

31 135 Measure valued Differentiation at Random Point

32 Increment distribution ~BM(0.1,0.5) Brownian motion Increment distribution ~P(20) Increment distribution ~CP(30,1,1,1,1,0.5) Left: Poisson process Right: Compound Poisson process

33 Increment distribution ~Gamma(10,1) Increment distribution ~VG(10,2,3) Left: Gamma process Right: Variance Gamma process

34 The geometric Brownian Motionn (GBM) Under the martingale measure, the GBM motion is dlogs(t) = (r 1 2 σ2 )dt + σdw (t) We aim at calculating the ρ, i.e. the sensitivity w.r.t. r for the two types of options The plain vanilla call option with payoff function g(s(t )) = e rt [S(T ) K] +. The digital option with payoff function g(s(t )) = e rt 1l {S(T )>K}.

35 Pathwise derivative dst r i = f r (St r i )dt + σ r (St r i )dw ti [. ] Dt r i+1 = Dt r i + f r (St r i ) + f r (St r i )Dt r i h + [. σ r (St r i ) + σ r (St r i )Dt r ] i hz. f r (St r ) = St r, f r (St r i ) = (r 1 2 σ2. ), σ r (St r i ) = 0, σ r (St r i ) = σ and therefore [ Dt r i+1 = Dt r i + St r i + (r 1 ] 1 2 σ2 )Dt r i n + [ σdt r ] 1 i n Z E (g(s T ) r = E ( g (S T )DT r )

36 Measure-valued derivative ( S(t i+1 ) S(t i ) N S(t i) r 1 ) 1 2 σ2, σ 2 S(t i ) 2 1 n n }{{}}{{} µ r σ 2 [S(t i+1 ) S(t i )] + = µ r + Rayleigh(σ ) [S(t i+1 ) S(t i )] = µ r Rayleigh(σ ) c r = µ r σ 2π = S(t i ) 1 n σs(t i ) = 1 n 2π 1 σ 2πn We consider the payoff function g(s T ) = e rt [S(T ) K] +. For the derivative we have to simulate E[g(S T )] r = c r ( E[g(S + T )] E[g(S T )]).

37 GBM: Plain vanilla call option ρ variance computational time in sec IPA e FD e MVD e MVDe e

38 GBM: Digital option ρ variance computational time in sec FD e MVD MVDe

39 Poisson: Plain vanilla option Sensitivity w.r.t. λ variance comp. time in sec FD e MVD MVDe

40 Poisson: Digital option Sensitivity w.r.t. λ variance comp. time in sec FD e MVD MVDe

41 Gamma: Plain vanilla Sensitivity w.r.t b variance comp. time in sec FD e MVD MVDe

42 Gamma: Digital Sensitivity w.r.t b variance computational time in sec FD e MVD MVDe