Algorithmic Trading: Optimal Control PIMS Summer School

Transcription

1 Algorihmic Trading: Opimal Conrol PIMS Summer School Sebasian Jaimungal, U. Torono Álvaro Carea,U. Oxford many hanks o José Penalva,(U. Carlos III) Luhui Gan (U. Torono) Ryan Donnelly (Swiss Finance Insiue, EPFL) Damir Kinzebulaov (U. Laval) Jason Ricci (Morgan Sanley) July, 2016 (c) S. Jaimungal, 2016 Algo Trading July, / 21

2 is concerned wih maximizing / minimizing a performance crieria where he crieria is affeced by fuure unknown noise in he sysem, as well as he acions of he conroller / agen (c) S. Jaimungal, 2016 Algo Trading July, / 21

3 We aim o solve he problem [ H(x) = sup E u A G(X x,u T ) }{{} erminal reward T + 0 F (s, Xs x,u, u s ) ds } {{ } running reward/penaly where u = (u ) 0 is he conrol process and he agen chooses i A is he admissible se of conrols e.g., exclude doubling sraegies X u = (X u ) 0 is he conrolled process, which he agen parially conrols, and saisfies he SDE dx u = µ(, X u, u ) d + σ(, X u, u ) dw, X 0 = x (c) S. Jaimungal, 2016 Algo Trading July, / 21

4 Trick is o inroduce a larger class of problems indexed by ime The value funcion is defined as [ T H(, x) = sup E,x G(XT u ) + F (s, Xs u, u s ) ds u A where E,x [ means expecaion condiional on X = x. Prove a dynamic programming principle and he corresponding dynamic programming equaion (c) S. Jaimungal, 2016 Algo Trading July, / 21

5 We will flow an arbirary admissible conrol and re-wrie he value funcion recursively [ E,X u G(XT u)+ T F(s,Xu s,u s )ds {}}{ X u 4 X u Time (c) S. Jaimungal, 2016 Algo Trading July, / 21

6 Take an arbirary admissible conrol u A H u (, x) [ T = E,x G(XT u ) + F (s, Xs u, u s) ds [ T = E,x G(XT u ) + F (s, Xs u, u s) ds + F (s, Xs u, u s) ds [ [ T = E,x E,X u G(XT u ) + F (s, Xs u, us) ds + F (s, Xs u, us) ds (by ieraed expecaion) [ = E,x H u (, X u ) + F (s, Xs u, us) ds (by defn) (c) S. Jaimungal, 2016 Algo Trading July, / 21

7 However, H(, x) H u (, x), wih equaliy if u = u, hence H u (, x) E,x [H(, X u ) + F (s, Xs u, u s) ds sup E,x [H(, X u ) + F (s, Xs u, u s) ds u A and so H(, x) sup E,x [H(, X u ) + F (s, Xs u, u s) ds u A (c) S. Jaimungal, 2016 Algo Trading July, / 21

8 Take an ε-opimal conrol υ ε a conrol ha performs beer han H(, x) ε, bu of course no as good as H(, x), i.e., such ha H(, x) H υε (, x) H(, x) ε Modify he ε-opimal conrol beween and by an arbirary conrol u, i.e., define υ ε by υ ε := u 1 + υ ε 1 > (c) S. Jaimungal, 2016 Algo Trading July, / 21

9 Then, H(, x) H υε (, x) [ = E,x H υε (, X υε ) + F (s, Xs υε, υ s ε ) ds = E,x [H υε (, X u ) + F (s, Xs u, u s) ds E,x [H(, X u ) ε + F (s, Xs u, u s) ds Hence, aking ε 0, and since u is arbirary by ieraed expecaions by he modified sraegy ε-opimal conrol H(, x) sup E,x [H(, X u ) + F (s, Xs u, u s) ds u A (c) S. Jaimungal, 2016 Algo Trading July, / 21

10 Puing boh inequaliies ogeher we arrive a he dynamic programming principle (DPP) H(, x) = sup E,x [H(, X u ) + F (s, Xs u, u s ) ds u A The dynamic programming equaion (DPE) is he infiniesimal version of his principle (c) S. Jaimungal, 2016 Algo Trading July, / 21

11 Take in he DPP o be he following = T inf {s > : (s, X u s x ) / [0, h) [0, ɛ)} X u Time ha is, eiher 1. an amoun of ime h passes, and he processes has deviaed by less han ɛ, or 2. he process deviaes by ɛ and we sop (c) S. Jaimungal, 2016 Algo Trading July, / 21

12 From he DPP H(, x) sup E,x [H(, X u ) + u A F (s, X u s, u s) ds Hence, for a consan sraegy υ on he inerval [, H(, x) E,x [H(, X υ ) + F (s, Xs υ, υ) ds Applying Iô s lemma o he value funcion H(, X υ ) = H(, X ) + where he generaor is L υ + ( + L υ s ) H(s, X υ s ) ds xh(s, X υ s ) σ(s, X υ s, υ) dw s, = µ(, x, υ) x σ2 (, x, υ) xx (c) S. Jaimungal, 2016 Algo Trading July, / 21

13 We hen have, H(, x) E,x [H(, X ) + ( + Ls υ ) H(s, Xs υ ) ds + xh(s, Xs υ ) σ(s, Xs υ, υ) dw s + F (s, X υ s, υ) ds Since X υ x < ɛ on [,, he sochasic inegral is indeed a maringale and we have { } H(, x) E,x [H(, x) + ( + Ls υ ) H(s, Xs υ ) + F (s, Xs υ, υ) ds (c) S. Jaimungal, 2016 Algo Trading July, / 21

14 Then, [ 1 { } 0 lim E,x ( + Ls υ ) H(s, Xs υ ) + F (s, Xs υ, υ) ds h 0 h = ( + L υ ) H(, x) + F (, x, υ) which follows b/c (i) as h 0, = + h a.s. since he process will no hi he barrier of ɛ in exremely shor periods of ime, (ii) he condiion ha X u x ɛ, which implies ha if he process does hi he barrier i is bounded, (iii) he Mean-Value Theorem allows us o wrie ω s ds = ω, and lim h 0 1 h +h (iv) he process sars a X υ = x. (c) S. Jaimungal, 2016 Algo Trading July, / 21

15 This inequaliy holds for every consan υ, and herefore H(, x) + sup {L υ H(, x) + F (, x, υ)} 0 υ (c) S. Jaimungal, 2016 Algo Trading July, / 21

16 Nex, we show he opposie inequaliy, i.e., ha H(, x) + sup {L υ H(, x) + F (, x, υ)} 0 υ We do his by conradicion... assume ( 0, x 0 ) s.., H( 0, x 0 ) + sup {L υ H( 0, x 0 ) + F ( 0, x 0, υ)} < 0 (1) υ Then, define a modificaion ϕ of he value funcion H via ϕ(, x) = H(, x) + ɛ ( ( 0 ) 2 + (x x 0 ) 4) his lies above he value funcion, bu equals i a ( 0, x 0 ), and is differeniable enough o apply Iô s lemma (c) S. Jaimungal, 2016 Algo Trading July, / 21

17 Noe also ha ϕ( 0, x 0) = H( 0, x 0), xϕ( 0, x 0) = xh( 0, x 0), xxϕ( 0, x 0) = xxh( 0, x 0) Therefore, from (1), we have ha ϕ( 0, x 0) + sup {L υ ϕ( 0, x 0) + F ( 0, x 0, υ)} < 0 υ and if he Hamilonian sup υ {L υ H(, x) + F (, x, υ)} is coninuous, hen a neighbourhood N r = ( 0 r, 0 + r) (x 0 r, x 0 + r) s.. ϕ(, x) + sup {L υ ϕ(, x) + F (, x, υ)} < 0 (2) υ for all (, x) N r (c) S. Jaimungal, 2016 Algo Trading July, / 21

18 Define η = max (ϕ H)(, x) > 0 (,x) N r Take an arbirary conrol u A and define he sopping ime Since X is coninuous = inf{s > 0 : X u s / N r } X u N r and herefore ϕ(, X u ) η + H(, X u ) (3) (c) S. Jaimungal, 2016 Algo Trading July, / 21

19 Apply Iô s lemma o ϕ o find ϕ(, X u ) = ϕ( 0, x 0)+ 0 ( +L u )ϕ(s, X u s ) ds+ 0 xϕ(s, X u s ) σ(s, X u s, u) dw s Therefore, V ( 0, x 0) = ϕ( 0, x 0) = E 0,x 0 [ϕ(, X u ) 0 ( + L u )ϕ(s, Xs u ) ds From (2), for (, x) N r ψ(, x) = ϕ(, x) + sup {L υ ϕ(, x) + F (, x, υ)} < 0 υ so ha ϕ(, x) + L u ϕ(, x) + F (, x, u ) ψ(, x) < 0 (c) S. Jaimungal, 2016 Algo Trading July, / 21

20 Therefore, we have V ( 0, x 0) = ϕ( 0, x 0) = E 0,x 0 [ϕ(, X u ) E 0,x 0 [ϕ(, X u ) ( + L u )ϕ(s, Xs u ) ds (F (s, Xs u, u s) ψ(s, Xs u )) ds On boundary N r, ϕ dominaes H, i.e. from (3), we have V ( 0, x 0) E 0,x 0 [η + H(, X u ) + η + E 0,x 0 [H(, X u ) (F (s, Xs u, u s) ψ(s, Xs u )) ds F (s, Xs u, u s) ds > E 0,x 0 [H(, X u ) + F (s, Xs u, u s) ds 0 This violaes he DPP! (since ψ < 0) (c) S. Jaimungal, 2016 Algo Trading July, / 21

21 Hence we obain he Dynamic Programming Equaion (DPE), aka Hamilon-Jacobi-Bellman (HJB) equaion H(, x) + sup {L υ H(, x) + F (, x, υ)} = 0 υ This is a non-linear PDE ha he value funcion mus saisfy I is no clear ha if we solve his PDE, he soluion is he value funcion! This requires a verificaion heorem (c) S. Jaimungal, 2016 Algo Trading July, / 21