Introduction to Algorithmic Trading Strategies Lecture 4

Transcription

1 Introduction to Algorithmic Trading Strategies Lecture 4 Optimal Pairs Trading by Stochastic Control Haksun Li haksun.li@numericalmethod.com

2 Outline Problem formulation Ito s lemma Dynamic programming Hamilton-Jacobi-Bellman equation Riccati equation Integrating factor 2

3 Reference Optimal Pairs Trading: A Stochastic Control Approach. Mudchanatongsuk, S., Primbs, J.A., Wong, W. Dept. of Manage. Sci. & Eng., Stanford Univ., Stanford, CA. 3

4 Basket Creation vs. Trading In lecture 3, we discussed a few ways to construct a mean-reverting basket. In this and the next lectures, we discuss how to trade a mean-reverting asset, if such exists. 4

5 Stochastic Control We model the difference between the log-returns of two assets as an Ornstein-Uhlenbeck process. We compute the optimal position to take as a function of the deviation from the equilibrium. This is done by solving the corresponding the Hamilton-Jacobi-Bellman equation. 5

6 Formulation Assume a risk free asset M t, which satisfies dm t = rm t dt Assume two assets,a t and B t. Assume B t follows a geometric Brownian motion. db t = μb t dt + σb t dz t x t is the spread between the two assets. x t = log A t log B t 6

7 Ornstein-Uhlenbeck Process We assume the spread, the basket that we want to trade, follows a mean-reverting process. dx t = k θ x t dt + ηdω t θ is the long term equilibrium to which the spread reverts. k is the rate of reversion. It must be positive to ensure stability around the equilibrium value. 7

8 Instantaneous Correlation Let ρ denote the instantaneous correlation coefficient between z and ω. E dω t dz t = ρdt 8

9 Univariate Ito s Lemma Assume dx t = μ t dt + σ t db t f t, X t is twice differentiable of two real variables We have df t, X t = f t + μ t f + σ t x f x 2 dt + σ t f x db t 9

10 Log example For G.B.M., dx t = μx t dt + σx t dz t, d log X t =? f x f t = 0 = log x f x = 1 x 2 f x 2 = 1 x 2 d log X t = μx t 1 + σx t X t X t 2 dt + σx t 1 X t db t = μ σ2 2 dt + σdb t 10

11 Multivariate Ito s Lemma Assume X t = X 1t, X 2t,, X nt is a vector Ito process f x 1t, x 2t,, x nt is twice differentiable We have df X 1t, X 2t,, X nt = n i=1 f X 1t, X 2t,, X nt dx i t n x i 2 n j=1 x i x j i=1 f X 1t, X 2t,, X nt d X i, X j t 11

12 Multivariate Example log A t = x t + log B t A t = exp x t + log B t A t x t = exp x t + log B t = A t A t B t = exp x t + log B t 1 B t = A t B t 2 A t x t 2 = A t x t = A t 2 A t B t 2 = B t 2 A t = B t x t B t A t B t = 0 A t x t = A t B t = A t B t 12

13 What is the Dynamic of Asset A t? A t = A t dx x t + A t db B t A t t 2 x 2 dx t A t B t x dx t db t = A t dx t + A t B t db t A t dx t 2 + A t B t dx t db t = A t k θ x t dt + ηdω t 1 2 A tη 2 dt + A t B t ρησb t dt + A t B t μb t dt + σb t dz t + 13

14 Dynamic of Asset A t A t = A t k θ x t dt + ηdω t + A t μdt + σdz t A tη 2 dt + A t ρησdt = A t k θ x t + μ η2 + ρησ dt + A t ηdω t + A t σdz t = A t μ + k θ x t η2 + ρησ dt + σdz t + ηdω t 14

15 Notations V t : the value of a self-financing pairs trading portfolio h t :the portfolio weight for stock A h t = h t :the portfolio weight for stock B 15

16 Self-Financing Portfolio Dynamic dv t V t = = h t da t A t + h t db t B t + dm t M t h t μ + k θ x t η2 + ρησ dt + σdz t + ηdω t h t μdt + σdz t + rdt = h t k θ x t η2 + ρησ dt + ηdω t + rdt = h t k θ x t η2 + ρησ + r dt + h t ηdω t 16

17 Power Utility Investor preference: U x = x γ x 0 0 < γ < 1 17

18 Problem Formulation max h t E V T γ, s.t., V 0 = v 0, x 0 = x 0 dx t = k θ x t dt + ηdω t dv t = h t dx t = h t k θ x t dt + h t ηdω t Note that we simplify GBM to BM of V t, and remove some constants. 18

19 Dynamic Programming Consider a stage problem to minimize (or maximize) the accumulated costs over a system path. Cost = c t=0 c t s11 2 S s0 1 s12 S22 4 s3 5 5 s12 s t = 0 t = 1 t = 2 time

20 Dynamic Programming Formulation State change: x k+1 = f k k: time x k : state x k, u k, ω k u k : control decision selected at time k ω k : a random noise N 1 Cost: g N x N + k=0 g k x k, u k, ω k Objective: minimize (maximize) the expected cost. We need to take expectation to account for the noise, ω k. 20

21 Principle of Optimality Let π = μ 0, μ 1,, μ N 1 be an optimal policy for the basic problem, and assume that when using π, a give state x i occurs at time i with positive probability. Consider the sub-problem whereby we are at x i at time i and wish to minimize the cost-to-go from time i to time N. N 1 E g N x N + k=0 g k x k, u k, ω k Then the truncated policy μ i, μ i+1,, μ N 1 is optimal for this sub-problem. 21

22 Dynamic Programming Algorithm For every initial state x 0, the optimal cost J x k of the basic problem is equal to J 0 x 0, given by the last step of the following algorithm, which proceeds backward in time from period N 1 to period 0: J N x N = g N x N J k x k = min u k E g k x k, u k, ω k + J k+1 f k x k, u k, ω k 22

23 Value function Terminal condition: G T, V, x = V γ DP equation: G t, V t, x t G t, V t, x t By Ito s lemma: = max E G t + dt, V t+dt, x t+dt h t = max E G t, V t, x t + ΔG h t ΔG = G t dt + G V dv + G x dx + 1 G 2 VV dv G 2 xx dx 2 + G Vx dv dx 23

24 Hamilton-Jacobi-Bellman Equation Cancel G t, V t, x t on both LHS and RHS. Divide by time discretization, Δt. Take limit as Δt 0, hence Ito. 0 = max h t E ΔG max E G t dt + G V dv h t + G x dx + 1 G 2 VV dv G 2 xx dx 2 + G Vx dv dx = 0 The optimal portfolio position is h t. 24

25 HJB for Our Portfolio Value max E G t dt + G V dv h t + G x dx + 1 G 2 VV dv G 2 xx dx 2 + G Vx dv dx = 0 max h t max h t E E G t dt + G V h t k θ x t dt + h t ηdω t + G x dx + 1 G 2 VV h t k θ x t dt + h t ηdω 2 t + 1 G 2 xx dx 2 + = 0 G Vx h t k θ x t dt + h t ηdω t dx G t dt + G V h t k θ x t dt + h t ηdω t + G x k θ x t dt + ηdω t + 1 G 2 VV h t k θ x t dt + h t ηdω 2 t + 1 G 2 xx k θ x t dt + ηdω 2 t + G Vx h t k θ x t dt + h t ηdω t k θ x t dt + ηdω t = 0 25

26 Taking Expectation All ηdω t disapper because of the expectation operator. Only the deterministic dt terms remain. Divide LHR and RHS by dt. max h t G t + G V h t k θ x t + G x k θ x t G VV h t η G xxη 2 +G Vx h t η 2 = 0 26

27 Dynamic Programming Solution Solve for the cost-to-go function, G t. Assume that the optimal policy is h t. 27

28 First Order Condition Differentiate w.r.t. h t. G V k θ x t + h t G VV η 2 + G Vx η 2 = 0 h t = G V k θ x t +G Vx η 2 G VV η 2 In order to determine the optimal position, h t, we need to solve for G to get G V, G Vx, and G VV. 28

29 The Partial Differential Equation (1) G t G V G V k θ x t +G Vx η 2 G VV η 2 k θ x t + G x k θ x t + 1 G 2 VVη 2 G V k θ x t +G Vx η 2 G VV η G xxη 2 G Vx η 2 G V k θ x t +G Vx η 2 G VV η = 0 29

30 The Partial Differential Equation (2) G t G V k θ x t G V k θ x t +G Vx η 2 G VV η G x k θ x t + G V k θ x t +G Vx η 2 2 G VV η G xxη 2 G Vx G V k θ x t +G Vx η 2 G VV = 0 30

31 Dis-equilibrium Let b = k θ x t. Rewrite: G t G V b G Vb+G Vx η 2 + G G VV η 2 x b G G V b+g Vx η 2 Vx = 0 G VV Multiply by G VV η 2. G V b+g Vx η 2 2 G VV η 2 G t G VV η 2 G V b G V b + G Vx η 2 + G x bg VV η G Vb + G Vx η G xxg VV η 4 G Vx η 2 G V b + G Vx η 2 = G xxη 2 31

32 Simplification Note that G V b G V b + G Vx η G 2 Vb + G Vx η 2 2 G Vx η 2 G V b + G Vx η 2 = 1 G 2 Vb + G Vx η 2 2 The PDE becomes G t G VV η 2 + G x bg VV η G 2 xxg VV η G Vb + G Vx η 2 2 = 0 32

33 The Partial Differential Equation (3) G t G VV η 2 + G x bg VV η G xxg VV η G Vb + G Vx η 2 2 = 0 33

34 Ansatz for G G t, V, x G T, V, x f T, x = 1 = f t, x V γ = V γ G t = V γ f t G V = γv γ 1 f G VV = γ γ 1 V γ 2 f G x = V γ f x G Vx = γv γ 1 f x G xx = V γ f xx 34

35 Another PDE (1) V γ f t γ γ 1 V γ 2 fη 2 + V γ f x bγ γ 1 V γ 2 fη Vγ f xx γ γ 1 V γ 2 fη γvγ 1 fb + γv γ 1 f x η 2 2 = 0 Divide by γ γ 1 η 2 V 2γ 2. f t f + f x bf f xxfη 2 γ 2 γ 1 f b η + f xη 2 = 0 35

36 Ansatz for f ff t + bff x η2 ff xx γ 2 γ 1 b η f + ηf x 2 = 0 f t, x = g t exp xβ t + x 2 α t = g exp xβ + x 2 α f t = g t exp xβ + x 2 α + g exp xβ + x 2 α xβ t + x 2 α t f x = g exp xβ + x 2 α β + 2xα f xx = g exp xβ + x 2 α β + 2xα 2 + g exp xβ + x 2 α 2α f x f = β + 2αx 36

37 Boundary Conditions f T, x = g T exp xβ T + x 2 α T = 1 g T = 1 α T = 0 β T = 0 37

38 Yet Another PDE (1) g exp xβ + x 2 α g t exp xβ + x 2 α + g exp xβ + x 2 α xβ t + x 2 α t + bg exp xβ + x 2 α g exp xβ + x 2 α β + 2xα η2 g exp xβ + x 2 α g exp xβ + x 2 α β + 2xα 2 + g exp xβ + x 2 α 2α γ 2 γ 1 b η g exp xβ + x2 α + ηg exp xβ + x 2 α Divide by g exp xβ + x 2 α exp xβ + x 2 α. β + 2xα 2 = 0 g t + g xβ t + x 2 α t + bg β + 2xα η2 g β + 2xα 2 + g 2α γ 2 γ 1 b + η β + 2xα η 2 = 0 g t + g xβ t + x 2 α t + bg β + 2xα η2 g β + 2xα 2 + η 2 gα γ 2 γ 1 b + η β + 2xα η 2 = 0 38

39 Yet Another PDE (2) λ = γ 2 γ 1 g t + g xβ t + x 2 α t + bg β + 2xα η2 g β + 2xα 2 + η 2 gα λg b + η β + 2xα η = 0 39

40 Expansion in x g t + gxβ t + gx 2 α t + bgβ + 2xαbg η2 g β 2 + 4x 2 α 2 + 4xαβ + η 2 gα λg b2 η 2 + η2 β 2 + 4η 2 x 2 α 2 + 2bβ + 4bxα + 4η 2 xαβ = 0 g t + gxβ t + gx 2 α t + k θ x gβ + 2xαk θ x g η2 g β 2 + 4x 2 α 2 + 4xαβ + η 2 gα λg k2 θ x 2 η 2 + η 2 β 2 + 4η 2 x 2 α 2 + 2k θ x β + 4k θ x xα + 4η 2 xαβ = 0 g t + gxβ t + gx 2 α t + kgβθ kgβx + 2xαkgθ 2αkgx η2 gβ 2 + 2η 2 gx 2 α 2 + 2η 2 gxαβ + η 2 gα λg η 2 k2 θ λg η 2 k2 θx λg η 2 k2 x 2 λgη 2 β 2 4λgη 2 x 2 α 2 2λgkβθ + 2λgkβx 4λgkθxα + 4λgkx 2 α 4λgη 2 xαβ = 0 40

41 Grouping in x g t + kgβθ η2 gβ 2 + η 2 gα λg η 2 k2 θ 2 λgη 2 β 2 2λgkβθ + gβ t kgβ + 2αkgθ + 2η 2 gαβ + 2 λg η 2 k2 θ + 2λgkβ 4λgkθα 4λgη 2 αβ x + gα t 2αkg + 2η 2 gα 2 λg η 2 k2 4λgη 2 α 2 + 4λgkα x 2 = 0 41

42 The Three PDE s (1) gα t 2αkg + 2η 2 gα 2 λg η 2 k2 4λgη 2 α 2 + 4λgkα = 0 gβ t kgβ + 2αkgθ + 2η 2 gαβ + 2 λg η 2 k2 θ + 2λgkβ 4λgkθα 4λgη 2 αβ = 0 g t + kgβθ η2 gβ 2 + η 2 gα λg η 2 k2 θ 2 λgη 2 β 2 2λgkβθ = 0 42

43 PDE in α α t + 2η 2 4λη 2 α 2 + 4λk 2k α λ η 2 k2 = 0 α t = λ η 2 k2 + 2k 1 2λ α + 2η 2 2λ 1 α 2 43

44 PDE in β, α β t kβ + 2η 2 αβ + 2λkβ 4λη 2 αβ 4λkθα + 2 λ η 2 k2 θ + 2αkθ = 0 β t = k 2η 2 α 2λk + 4λη 2 α β + 4λkθα 2 λ η 2 k2 θ 2αkθ 44

45 PDE in β, α, g g t + kgβθ η2 gβ 2 + η 2 gα λg η 2 k2 θ 2 λgη 2 β 2 2λgkβθ = 0 g t = kgβθ 1 2 η2 gβ 2 η 2 gα + λg η 2 k2 θ 2 + λgη 2 β 2 + 2λgkβθ g t = g kβθ 1 2 η2 β 2 η 2 α + λ η 2 k2 θ 2 + λη 2 β 2 + 2λkβθ 45

46 Riccati Equation A Riccati equation is any ordinary differential equation that is quadratic in the unknown function. α t = λ η 2 k2 + 2k 1 2λ α + 2η 2 2λ 1 α 2 α t = A 0 + A 1 α + A 2 α 2 46

47 Solving a Riccati Equation by Integration Suppose a particular solution, α 1, can be found. α = α is the general solution, subject to some z boundary condition. 47

48 Particular Solution Either α 1 or α 2 is a particular solution to the ODE. This can be verified by mere substitution. α 1,2 = A 1± A 2 1 4A 2 A 0 2A 2 48

49 z Substitution Suppose α = α z. 1 z = A 0 + A 1 α z + A 2 α z = A 0 + A 1 α 1 + A 1 1 z + A 2α A 2 1 z 2 + 2A 2 = A 0 + A 1 α 1 + A 1 1 z + A 2α A 2 1 z 2 + 2A 2 = A 0 + A 1 α 1 + A 2 α A 1+2α 1 A 2 z 2 + A 2 z 2 α 1 z α 1 z goes to 0 by the definition of α 1 49

50 Solving z 1 z = A 1+2α 1 A 2 z + A 2 z 2 1 z = A 1+2α 1 A 2 z 2 z 1 st order linear ODE + A 2 z 2 z + A 1 + 2α 1 A 2 z = A 2 z t = A 2 A 1 +2α 1 A 2 + Cexp A 1 + 2α 1 A 2 t 50

51 Solving for α α = α 1 + boundary condition: α T = 0 α A2 A1+2α1A2 +Cexp A 1+2α 1 A 2 t 1 A2 +Cexp = 0 A1+2α1A2 A 1+2α 1 A 2 T Cexp A 1 + 2α 1 A 2 T = 1 α 1 + C = exp A 1 + 2α 1 A 2 T A 2 A 2 A 1 +2α 1 A 2 A 1 +2α 1 A 2 1 α 1 51

52 α Solution (1) α = α 1 + = α A2 A1+2α1A2 +Cexp A 1+2α 1 A 2 t A2 A1+2α1A2 +exp A 1+2α 1 A 2 T = α 1 + = α 1 + A2 A1+2α1A2 +exp A 1+2α 1 A 2 1 A2 A1+2α1A2 1 1 T t α 1 A 1 +2α 1 A 2 α1 exp A 1+2α 1 A 2 t A2 A1+2α1A2 1 α1 α 1 A 2 +exp A 1 +2α 1 A 2 T t α 1 A 2 A 1 2α 1 A 2 52

53 α Solution (2) α = α 1 + = α 1 = α α 1 A 1 +2α 1 A 2 α 1 A 2 +exp A 1 +2α 1 A 2 T t A 1 α 1 A 2 A 1 +2α 1 A 2 A 2 +exp A 1 +2α 1 A 2 A1 A2 +2α 1 T t 1+ A 1 α1a2 +1 exp A 1+2α 1 A 2 A1 α1 +A 2 T t 53

54 α Solution (3) α t = k 2η γ γ γ exp 2k 1 γ T t 54

55 Solving β β t = k 2η 2 α 2λk + 4λη 2 α β + 4λkθα 2 λ η 2 k2 θ 2αkθ Let τ = T t β τ = β T t β τ τ = β t τ β τ τ = β t τ = k 2η 2 α τ 2λk + 4λη 2 α τ β τ + 4λkθα τ 2 λ η 2 k2 θ 2α τ kθ β τ τ = k + 2η 2 α + 2λk 4λη 2 α β + 4λkθα + 2 λ η 2 k2 θ + 2αkθ β τ τ = 2λ 1 k + 2η 2 α 1 2λ β + 2αkθ 1 2λ + 2 λ η 2 k2 θ 55

56 First Order Non-Constant Coefficients β τ = B 1 β + B 2 B 1 τ = 2λ 1 k + 2η 2 α 1 2λ B 2 τ = 2αkθ 1 2λ + 2 λ η 2 k2 θ 56

57 Integrating Factor (1) β τ B 1 β = B 2 We try to find an integrating factor μ = μ τ s.t. d dτ μβ = μ dβ dτ + β dμ dτ = μb 2 Divide LHS and RHS by μβ. 1 β dβ dτ + 1 μ dμ = B 2 dτ β By comparison, B 1 = 1 μ dμ dτ 57

58 Integrating Factor (2) B 1 dτ = dμ μ μ = exp B 1 dτ μβ = μb 2 dτ + C = log μ + C β = β = μb 2dτ+C μ exp B 1 du B 2 dτ+c exp B 1 dτ 58

59 β Solution τ β = exp B 1 u du 0 0 exp τ τ 0 B 1 u du B 2 s ds + C β τ = τ τ s exp B 1 u du exp B 0 1 u du 0 0 B 2 s ds + C 59

60 B 1, B 2 τ B 1 s ds 0 I 2 = τ 0 τ = 2λ 1 k + 2η 2 1 2λ α s ds 0 s 0 exp B 1 u du B 2 s ds 60

61 β Solution β t = kθ η γ exp 2k 1 γ T t γ exp 1 2k 1 γ T t 61

62 Solving g g t = g kβθ 1 2 η2 β 2 η 2 α + λ η 2 k2 θ 2 + λη 2 β 2 + 2λkβθ Withα and β solved, we are now ready to solve g t. g t g = G t d ds log g t = g t g = G log g t = Gds + C T g t = Cexp Gdt = exp Gds 0 T g t = exp Gds t t exp Gds 0 62

63 Computing the Optimal Position h t = G V k θ x t +G Vx η 2 G VV η 2 = γvγ 1 f k θ x t +γv γ 1 f x η 2 γ γ 1 V γ 2 fη 2 = Vf k θ x t +Vf x η 2 γ 1 fη 2 = V f k θ x t +f x η 2 γ 1 η 2 f V = k θ x γ 1 η 2 t + f x f η2 = V k θ x 1 γ η 2 t + η 2 β + 2αx = V 1 γ k η 2 x t θ + 2αx + β 63

64 The Optimal Position h t = h t ~ k η 2 V t 1 γ k η 2 x t θ + 2α t x t + β t x t θ 64

65 P&L for Simulated Data The portfolio increases from $1000 to $4625 in one year. 65

66 Parameter Estimation Can be done using Maximum Likelihood. Evaluation of parameter sensitivity can be done by Monte Carlo simulation. In real trading, it is better to be conservative about the parameters. Better underestimate the mean-reverting speed Better overestimate the noise To account for parameter regime changes, we can use: a hidden Markov chain model moving calibration window 66