Relatos to Other Statstcal Methods Statstcal Data Aalyss wth Postve Defte Kerels Kej Fukuzu Isttute of Statstcal Matheatcs, ROIS Departet of Statstcal Scece, Graduate Uversty for Advaced Studes October 6-0, 008, Kyushu Uversty
Outle. Sple soothg ad RKHS. Relato to rado process
Outle. Sple soothg ad RKHS. Relato to rado process 3
Sple soothg X,Y ),..., X, Y ) : X R, Y R P: dfferetal operator o R Sple soothg: = ) + λ Y f X ) Pf x) dx f Roughess pealty 4
Laplaca ad Gree fucto Laplaca Self-adjot: f f f Δ f = + + L+ x x x f f x), g x) 0 Δ f x) g xdx ) = f x) Δg xdx ) [partal tegral] Gree fucto for Laplaca Δ Gx, ξ ) = δ x ξ).e. Gree fucto solves a dfferetal equato: ) f x) = G x, y) ϕ y) dy Δ f = ϕ gve ϕ. 5
Soothg pealty Regularzato ter Cosder fuctos o R for splcty o boudary)! α J f) = D f α! α! Lα! = α + L+ α =! f α α α!!! α + L+ α = α α Lα x x L x L L or of -th dervatve dx exaple = = ) f f f J f) = + + dx x x x x 6
Soothg ) + λ f = = 0 Y f X ) a J f) a 0) Expresso by Laplaca Partal tegral shows ) ) J f) = f, Δ f The soothg proble s expressed by = Y f X ) + λ f Af ) ), f L L where A = ) a Δ = 0 7
Case e.g. Two cases The Gree fucto s a postve defte kerel. The pealty ter s equal to the squared RKHS or. Case e.g. a 0 0 a = 0 0 f x) dx + f x) ) dx = f, f ) + f, Δf ) x ) f x) dx = f, Δ f ) x Sple soothg The fuctoal space s RKHS + polyoal of soe order The pealty ter s equal to the squared RKHS or of the projecto of f oto the RKHS. L L L 8
a 0 0 : RKHS regularzato Soluto = Varatoal calculus Y f X ) + λ f Af ) ), f = ) Y f x) δ x X ) + λaf = 0 Af = Y f x) δ x X ) λ = = = ) If we have the Gree fucto G for A.e. f ξ ) = Y f x)) δ x X ) G x, ξ) dx λ = Y f X )) G ξ, X ) λ ote: fx ) ukow L 9
The soluto s to have the for: f = cg, X ) = Plug t to the orgal proble: c R = ) j j j = j + λ, j= j Y c G X, X ) cc G X, X ) Q) By dfferetato, c= G+ λi) Y where G j = G X, X ) j Y = Y, K, Y ) T The soluto: T Y where g ), x = G x X ) f x) = G+λI) g x) 0
Gree fucto Theore If a0 0, a j 0 j ), the Gree fucto of A s a postve defte kerel. Proof. Sce Α s shft varat, so s G Gx, y) = Gx-y) ). Thus, = 0 By Fourer trasfor ) a Δ G z) = δ z) If a0 0, a j 0 j ), the Fourer verso s possble. Use Bocher s theore.
Regularzato by RKHS or Assue a0 0, a 0 G: Gree fucto of A. H G : RKHS w.r.t. G. ) + λ f = = 0 Y f X ) a J f) The soluto s gve by The pealty ter s, the, f = = cg, X ) The above regularzato s equvalet to the kerel rdge regresso = ) Y f X + λ f ) f H G
a 0 = 0: Sple soothg Th-plate sple = ) Y f X ) + λj f) f! α J f ) = D f α + L+ α = α! α! Lα! L The Gree fucto of s ot ecessarly postve defte, but codtoally postve defte). The fucto space for f s ad α B : D f L R ) α = ) J f) = 0 f P P - : Polyoals of degree at ost - 3
B = P H Let be decoposto by drect su. * Theore Meguet 979) If > /,the subspace H * s a RKHS wth er product! α = α! Lα! I partcular, the or s gve by H f = J f) * α α ) ) f, g = D f, D g = ) Δ f, g H * L L = ) Y f X ) + λj f) f g H* p P, = ) Y g X + p X + λ g H* ) )) 4
Soluto of sple soothg By the represeter theore, the soluto s to be of the for: By pluggg t, f x) = ck x X ) + bφ x) M l l = l= The soluto: 5
Outle. Sple soothg. Relato to rado process 6
Gaussa process A Gaussa process s a rado process rado varables wth dex Ω) such that for ay fte subset {t,..., t } of Ω, the rado vector X, K, ) s a Gaussa rado vector. Mea fucto Covarace fucto t X t A Gaussa process s uquely detered by the ea ad covarace fucto. X = X t,..., X ) t Rt, t) Rt, t) L Rt, t) Rt, t) Rt, t) L Rt, t) μx = μ t), K, μ t )), Σ X = M M O M Rt, t) Rt, t) L Rt, t) 7
Exaples 3 3 0 0 - - - - -3-3 -8-6 -4-0 4 6 8-4 -8-6 -4-0 4 6 8 σ = σ = 0.3 ea zero covarace fucto Rst,) = exp s t) σ Geerated by Matlab gpl toolbox Rasusse ad Wllas) 8
Rado process ad postve defte kerel Covarace fucto s a postve defte kerel Theore The covarace fucto Rs, t) of a rado process s a postve defte kerel. Q) For splcty, ea = 0., j= cc jr t, tj ) =, j= cc je[ Xt, X ] tj E cxt j cjxt E = = = cxt =, = ) 0 j A rado process o Ω deteres a RKHS o Ω. 9
Postve defte kerel defes Gaussa process ks,t): postve defte kerel o Ω. For ay fte subset t = t,, t ) of Ω, the Gra atrx Σ t = kt, t j ) ) s always postve sedefte. By Kologorov exteso theore, there s a Gaussa process wth dex set Ω such that X = X,..., X ) t t The covarace fucto = ks,t). 0
Statoary process ad shftvarat kerel Statoary case : rado process o R statoary process EX [ + X + ] = EXX [ ] tsh,, R ) t h s h t s covarace fucto s gve by Rts, ) Rt s) Postve defte kerel for a statoary process s gve by Kts, ) = Kt s) Bocher s theore Weer-Khche s theore covarace fucto of a statoary process o R s the verse Fourer trasfor of the power spectral.)
Refereces Wahba, G. Sple Models for Observatoal Data. CBMS-SF Regoal Coferece Seres Appled Matheatcs 59. SIAM. 990. Meguet, J. 979) Multvarate Iterpolato at Arbtrary Pots Made Sple. J. Appled Matheatcs ad Physcs ZAMP) 30, 9-304. Berlet, A. ad C. Thoas-Aga. Reproducg Kerel Hlbert Spaces Probablty ad Statstcs. Kluwer Acadec Publshers, 003. Gel fad, I.M. ad Vlek,.Ya. Geeralzed Fuctos Vol.4: Applcatos of Haroc Aalyss. Acadec Press. 964.