Relations to Other Statistical Methods Statistical Data Analysis with Positive Definite Kernels

Transcription

1 Relatos to Other Statstcal Methods Statstcal Data Aalyss wth Postve Defte Kerels Kej Fukuzu Isttute of Statstcal Matheatcs, ROIS Departet of Statstcal Scece, Graduate Uversty for Advaced Studes October 6-0, 008, Kyushu Uversty

2 Outle. Sple soothg ad RKHS. Relato to rado process

3 Outle. Sple soothg ad RKHS. Relato to rado process 3

4 Sple soothg X,Y ),..., X, Y ) : X R, Y R P: dfferetal operator o R Sple soothg: = ) + λ Y f X ) Pf x) dx f Roughess pealty 4

5 Laplaca ad Gree fucto Laplaca Self-adjot: f f f Δ f = + + L+ x x x f f x), g x) 0 Δ f x) g xdx ) = f x) Δg xdx ) [partal tegral] Gree fucto for Laplaca Δ Gx, ξ ) = δ x ξ).e. Gree fucto solves a dfferetal equato: ) f x) = G x, y) ϕ y) dy Δ f = ϕ gve ϕ. 5

6 Soothg pealty Regularzato ter Cosder fuctos o R for splcty o boudary)! α J f) = D f α! α! Lα! = α + L+ α =! f α α α!!! α + L+ α = α α Lα x x L x L L or of -th dervatve dx exaple = = ) f f f J f) = + + dx x x x x 6

7 Soothg ) + λ f = = 0 Y f X ) a J f) a 0) Expresso by Laplaca Partal tegral shows ) ) J f) = f, Δ f The soothg proble s expressed by = Y f X ) + λ f Af ) ), f L L where A = ) a Δ = 0 7

8 Case e.g. Two cases The Gree fucto s a postve defte kerel. The pealty ter s equal to the squared RKHS or. Case e.g. a 0 0 a = 0 0 f x) dx + f x) ) dx = f, f ) + f, Δf ) x ) f x) dx = f, Δ f ) x Sple soothg The fuctoal space s RKHS + polyoal of soe order The pealty ter s equal to the squared RKHS or of the projecto of f oto the RKHS. L L L 8

9 a 0 0 : RKHS regularzato Soluto = Varatoal calculus Y f X ) + λ f Af ) ), f = ) Y f x) δ x X ) + λaf = 0 Af = Y f x) δ x X ) λ = = = ) If we have the Gree fucto G for A.e. f ξ ) = Y f x)) δ x X ) G x, ξ) dx λ = Y f X )) G ξ, X ) λ ote: fx ) ukow L 9

10 The soluto s to have the for: f = cg, X ) = Plug t to the orgal proble: c R = ) j j j = j + λ, j= j Y c G X, X ) cc G X, X ) Q) By dfferetato, c= G+ λi) Y where G j = G X, X ) j Y = Y, K, Y ) T The soluto: T Y where g ), x = G x X ) f x) = G+λI) g x) 0

11 Gree fucto Theore If a0 0, a j 0 j ), the Gree fucto of A s a postve defte kerel. Proof. Sce Α s shft varat, so s G Gx, y) = Gx-y) ). Thus, = 0 By Fourer trasfor ) a Δ G z) = δ z) If a0 0, a j 0 j ), the Fourer verso s possble. Use Bocher s theore.

12 Regularzato by RKHS or Assue a0 0, a 0 G: Gree fucto of A. H G : RKHS w.r.t. G. ) + λ f = = 0 Y f X ) a J f) The soluto s gve by The pealty ter s, the, f = = cg, X ) The above regularzato s equvalet to the kerel rdge regresso = ) Y f X + λ f ) f H G

13 a 0 = 0: Sple soothg Th-plate sple = ) Y f X ) + λj f) f! α J f ) = D f α + L+ α = α! α! Lα! L The Gree fucto of s ot ecessarly postve defte, but codtoally postve defte). The fucto space for f s ad α B : D f L R ) α = ) J f) = 0 f P P - : Polyoals of degree at ost - 3

14 B = P H Let be decoposto by drect su. * Theore Meguet 979) If > /,the subspace H * s a RKHS wth er product! α = α! Lα! I partcular, the or s gve by H f = J f) * α α ) ) f, g = D f, D g = ) Δ f, g H * L L = ) Y f X ) + λj f) f g H* p P, = ) Y g X + p X + λ g H* ) )) 4

15 Soluto of sple soothg By the represeter theore, the soluto s to be of the for: By pluggg t, f x) = ck x X ) + bφ x) M l l = l= The soluto: 5

16 Outle. Sple soothg. Relato to rado process 6

17 Gaussa process A Gaussa process s a rado process rado varables wth dex Ω) such that for ay fte subset {t,..., t } of Ω, the rado vector X, K, ) s a Gaussa rado vector. Mea fucto Covarace fucto t X t A Gaussa process s uquely detered by the ea ad covarace fucto. X = X t,..., X ) t Rt, t) Rt, t) L Rt, t) Rt, t) Rt, t) L Rt, t) μx = μ t), K, μ t )), Σ X = M M O M Rt, t) Rt, t) L Rt, t) 7

18 Exaples σ = σ = 0.3 ea zero covarace fucto Rst,) = exp s t) σ Geerated by Matlab gpl toolbox Rasusse ad Wllas) 8

19 Rado process ad postve defte kerel Covarace fucto s a postve defte kerel Theore The covarace fucto Rs, t) of a rado process s a postve defte kerel. Q) For splcty, ea = 0., j= cc jr t, tj ) =, j= cc je[ Xt, X ] tj E cxt j cjxt E = = = cxt =, = ) 0 j A rado process o Ω deteres a RKHS o Ω. 9

20 Postve defte kerel defes Gaussa process ks,t): postve defte kerel o Ω. For ay fte subset t = t,, t ) of Ω, the Gra atrx Σ t = kt, t j ) ) s always postve sedefte. By Kologorov exteso theore, there s a Gaussa process wth dex set Ω such that X = X,..., X ) t t The covarace fucto = ks,t). 0

21 Statoary process ad shftvarat kerel Statoary case : rado process o R statoary process EX [ + X + ] = EXX [ ] tsh,, R ) t h s h t s covarace fucto s gve by Rts, ) Rt s) Postve defte kerel for a statoary process s gve by Kts, ) = Kt s) Bocher s theore Weer-Khche s theore covarace fucto of a statoary process o R s the verse Fourer trasfor of the power spectral.)

22 Refereces Wahba, G. Sple Models for Observatoal Data. CBMS-SF Regoal Coferece Seres Appled Matheatcs 59. SIAM Meguet, J. 979) Multvarate Iterpolato at Arbtrary Pots Made Sple. J. Appled Matheatcs ad Physcs ZAMP) 30, Berlet, A. ad C. Thoas-Aga. Reproducg Kerel Hlbert Spaces Probablty ad Statstcs. Kluwer Acadec Publshers, 003. Gel fad, I.M. ad Vlek,.Ya. Geeralzed Fuctos Vol.4: Applcatos of Haroc Aalyss. Acadec Press. 964.