Rough Paths and its Applications in Machine Learning

Transcription

1 Pah ignaure Machine learning applicaion Rough Pah and i Applicaion in Machine Learning July 20, 2017 Rough Pah and i Applicaion in Machine Learning

2 Pah ignaure Machine learning applicaion Hiory and moivaion Terry Lyon (1998), Differenial equaion driven by rough ignal, Rev. Ma. Iberoamericana 14. Originally formulaed o udy ochaic differenial equaion in a pah-wie manner. Rough pah heory heory of regulariy rucure Soluion o KPZ equaion (Marin Hairer, Field medal 2014). Now begin applied o oher hard problem in aiical phyic. Recenly, rough pah heory i finding applicaion in machine learning. Rough Pah and i Applicaion in Machine Learning

3 Pah ignaure Machine learning applicaion Conrolled differenial equaion Conider he following differenial equaion: y = V (y ) x Fir-order approximaion o he oluion: For, [0, T ], y y = V (y ) V (y r ) x r dr x r dr = V (y ) (x x ) Rough Pah and i Applicaion in Machine Learning

4 Pah ignaure Machine learning applicaion Second-order approximaion y y = V (y r1 ) x r 1 dr 1 ( r1 V (y ) + ) = V (y r2 ) y r 2 dr 2 x r 1 dr 1 [ ] = V (y ) x r 1 dr 1 + ( r1 ) V (y r2 )V (y r2 ) x r 2 dr 2 x r 1 dr 1 [ ] V (y ) dx r1 [ + V (y )V (y ) ( r1 ) ] dx r2 dx r1 Rough Pah and i Applicaion in Machine Learning

5 Pah ignaure Machine learning applicaion Separaing ignal from vecor field In one dimenion: In d dimenion: y y V (y ) [x x ] + V (y )V (y ) [ ] 1 2 (x x ) 2 y y V (y ) [x x ] d ( + V (y )V (y ) r1 i,j=1 ) dx r (i) 2 dx r (j) 1 e i e j Rough Pah and i Applicaion in Machine Learning

6 Pah ignaure Machine learning applicaion 2-dimenional cae ( Given y = ( ) y (1) y (2) ) = [ V (1) 1 (y ) V (1) V (2) 1 (y ) V (2) 2 (y ) ] ) 2 (y ) (x (1) ( ), x (2) we have y y [ V (1) 1 (y ) V (1) 2 (y ) V (2) 1 (y ) V (2) 2 (y ) + V (y )V (y ) ] [ ([ ] x (1) x (1) x (2) x (2) r1 dx (1) r 2 dx r (1) 1 r1 dx (2) r 2 dx r (1) 1 r1 dx (1) r 2 dx r (2) 1 r1 dx (2) r 2 dx r (2) 1 ]). Rough Pah and i Applicaion in Machine Learning

7 Pah ignaure Machine learning applicaion 2-dimenional cae con. where [ r1 dx (1) r 2 dx r (1) 1 r1 dx (2) r 2 dx r (1) 1 x (k), r1 dx (1) r 2 dx r (2) 1 r1 dx (2) r 2 dx r (2) 1 ( = 1 2 and he Lévy-area i given by A = r1 x (1), x (2), x (1), ] ) 2 x (1), x (2), ) 2 x (2), ( } {{ } ymmeric par := x (k) x (k), k = 1, 2, dx r (1) 2 dx r (2) 1 r1 + 1 [ ] 0 A 2 A 0 }{{} ani ymmeric par dx (2) r 2 dx (1) r 1. Rough Pah and i Applicaion in Machine Learning

8 Pah ignaure Machine learning applicaion Green heorem A = 1 x (2) dx (1) + x (1) dx (2) 2 C Rough Pah and i Applicaion in Machine Learning

9 Pah ignaure Machine learning applicaion Signaure of a pah For all 0 T, S n (x), := ( 1, x,, x 2,,..., x n ), R R d ( R d R d) ( R d) n, where x k, i he convenional k-h ieraed inegral of he pah x over he inerval [, ]: d ( x k, = j 1,...,j k =1 <r 1< <r k < dx (j1) r 1 dx (j k ) r k ) e j1 e jk. Rough Pah and i Applicaion in Machine Learning

10 Pah ignaure Machine learning applicaion Chen Ideniy The ignaure i an elemen of a Lie group called he ep-n nilpoen group wih d generaor. I aifie S n (x), = S n (x),u S n (x) u,, u, [0, T ], u, where given a = ( 1, a 1,..., a n), b = ( 1, b 1,..., b n), group muliplicaion i performed by a b := ( 1, c 1,..., c n), c k = k a i b k i, 1 k n. E.g. (1, a 1, a 2 ) (1, b 1, b 2 ) = (1, a 1 + b 1, a 2 + b 2 + a 1 b 1 ). i=0 Rough Pah and i Applicaion in Machine Learning

11 Pah ignaure Machine learning applicaion Fracional Brownian moion: W H (i) Hur parameer: H (0, 1) (ii) Coninuou pah, no differeniable a.e. (iii) W W N (0, 2H ) (iv) Covariance funcion: R(, ) = 1 2 ( 2H + 2H 2H) (v) H = 1 2 : Sandard Brownian moion, R(, ) =. (vi) H > 1 2 : Incremen along dijoin inerval are poiively correlaed. (vii) H < 1 2 : Incremen along dijoin inerval are negaively correlaed. (viii) Neiher a Markov proce nor a maringale (unle H = 1 2 ) Rough Pah and i Applicaion in Machine Learning

12 Pah ignaure Machine learning applicaion Sample pah Rough Pah and i Applicaion in Machine Learning

13 Pah ignaure Machine learning applicaion Hölder coninuiy and rough pah Definiion A funcion f i aid o be α-hölder coninuou on an inerval [0, T ] if f () f () C α,, [0, T ]. Hölder coninuiy meaure how rough a funcion i. Fac: Fracional Brownian moion wih Hur parameer H i almo urely (H ε)-hölder coninuou for any ε > 0. Definiion Given 1 3 < α 1 2, X = ( 1, X,, X 2,) i an α-hölder rough pah if i aifie Chen ideniy, X i α-hölder coninuou and X 2 i 2α-Hölder coninuou. Rough Pah and i Applicaion in Machine Learning

14 Pah ignaure Machine learning applicaion Ió inegraion a rough pah inegraion T 0 Y dw = lim π 0 i Y i W i, i+1, where he limi i aken in L 2 (Ω) and no almo urely pah-wie becaue he pah are no regular enough. Even o, convergence in L 2 (Ω) relie on he fac W i a maringale, and ha Y i adaped o he filraion of W. Define W, Io := ( ( 1, A 1,, A 2 ) ), = 1, W,, W,r1 dw r1 Then given a Gubinelli derivaive Y, T 0 almo urely. Y dw = T 0 Y dw Io = lim π 0 Y i A 1 i, i+1 + Y A 2 i, i+1 Rough Pah and i Applicaion in Machine Learning

15 Pah ignaure Machine learning applicaion Properie of he ignaure The ignaure ha more or le a one-o-one relaion wih i pah (ee T. Lyon and B. Hambly, Uniquene for he ignaure of a pah of bounded variaion and he reduced pah group, 2010). I i a graded ummary of he daa ream encoded in he pah. Ieraed inegral capure non-linear apec of he pah. Form a naural bai for funcional on daa ream. I provide a rich e of feaure ha can be ued in a machine learning pipeline. Rough Pah and i Applicaion in Machine Learning

16 Pah ignaure Machine learning applicaion Applicaion Finance: J. Field, L. Gyurkó, M. Konkowki and T. Lyon (2014), Exracing informaion from he ignaure of a financial daa ream, arxiv: v2. Sound compreion: T. Lyon and N. Sidorova (2005), Sound compreion: a rough pah approach, In Proceeding of he 4h inernaional ympoium on Informaion and communicaion. Idenifying paern in MEG can ec. I. Chevyrev and A. Kormilizin (2016), A Primer on he Signaure Mehod in Machine Learning, arxiv: v1. Rough Pah and i Applicaion in Machine Learning

17 Pah ignaure Machine learning applicaion Chinee handwriing recogniion SCUT gpen: Online Chinee handwriing recogniion ofware Began a a collaboraion beween Terry Lyon and Ben Graham (Univeriy of Warwick) App developed by HCII-Lab in Souh China Univeriy of Technology Sae of he ar: Won fir place in ICDAR2013 compeiion wih an error rae of 2.61% (Second place: 3.13%, Human error: 4.81%). Combine rough pah heory and a deep convoluional neural nework. Ue fir 3 level of he ignaure of he pah. Ben Graham (2013), Spare array of ignaure for online characer recogniion, arxiv: v2. Rough Pah and i Applicaion in Machine Learning

18 Pah ignaure Machine learning applicaion Concluion and rambling Texbook: Peer Friz and Marin Hairer (2014), A Coure on Rough Pah, Springer. Terry Lyon, M. Caruana, and T. Lévy (2007), Differenial equaion driven by Rough Pah, Springer. Fuure direcion: Applicaion o ochaic conrol and reinforcemen learning: (i) Exend conrol heory o dynamical yem perurbed by coloured noie. (ii) Find efficien Mone-Carlo cheme o compue opimal pah and conrol rajecorie. Rough Pah and i Applicaion in Machine Learning