MAXIMUM ENTROPIES COPULAS

MAXIMUM ENTROPIES COPULAS Doriano-Boris Pougaza & Ali Mohammad-Djafari Groupe Problèmes Inverses Laboratoire des Signaux et Systèmes (UMR 8506 CNRS - SUPELEC - UNIV PARIS SUD) Supélec, Plateau de Moulon, 91192 Gif-sur-Yvette, France. doriano-boris.pougaza@u-psud.fr http://users.aims.ac.za/ ~ doriano Chamonix, France, July 4-9, 2010

STATISTICS & TOMOGRAPHY Tomography : Given two projections horizontal and vertical f 1 (x) and f 2 (y), find image f(x, y) Statistics : Given two marginals pdfs f 1 (x) et f 2 (y), find the joint distribution f(x, y) Two equivalent ill-posed Inverse Problems : Infinite number of solutions D-B. POUGAZA & al. () MAXIMUM & COPULA MaxEnt 2010 2 / 21

Content 1 OVERVIEW ABOUT COPULAS IN STATISTICS 2 MAXIMUM ENTROPIES COPULAS 3 NEW FAMILIES OF COPULAS D-B. POUGAZA & al. () MAXIMUM & COPULA MaxEnt 2010 3 / 21

OVERVIEW ABOUT COPULAS IN STATISTICS WHAT IS A COPULA? 1 SIMPLE DEFINITION : A copula is a multivariate probability distribution function defined on [0, 1] n whose marginals are uniform. 2 IN THE STATISTICS LITERATURE copula is a tool to link a multivariate distribution function to its marginal distributions. f(x, y) = f 1 (x)f 2 (y)ω(x, y). 3 POWERFUL TOOLS IN MODELING Mostly used in Finance and Environmental Sciences (C. Genest & MacKay, 1986 ; R.M Cooke, 1997 ; P. Embrechts, 2003) Offer several choices to model dependency between variables (H. Joe, 1997 ; R.B. Nelsen, 2006) D-B. POUGAZA & al. () MAXIMUM & COPULA MaxEnt 2010 4 / 21

OVERVIEW ABOUT COPULAS IN STATISTICS COPULA DEFINITION A bivariate copula C is a function C : [0, 1] [0, 1] [0, 1] (u, v) C(u, v) with the following properties : 1 C(u, 0) = 0 = C(0, v), 2 C(u, 1) = u and C(1, v) = v, 3 C(u 2, v 2 ) C(u 2, v 1 ) C(u 1, v 2 ) + C(u 1, v 1 ) 0, for all 0 u 1 u 2 1 and 0 v 1 v 2 1, 4 C(u, v) min {u, v}, 5 C(u, v) max {u + v 1, 0}. D-B. POUGAZA & al. () MAXIMUM & COPULA MaxEnt 2010 5 / 21

OVERVIEW ABOUT COPULAS IN STATISTICS SKLAR s THEOREM (1959) Let F be a two-dimensional distribution function with marginal distributions functions F 1 and F 2. Then there exists a copula C such that : F (x, y) = C(F 1 (x), F 2 (y)). (1) If the marginal functions are continuous, then the copula C is unique, and is given by C(u, v) = F (F 1 1 (u), F 1 2 (v)). (2) Otherwise C is uniquely determined on Ran (F 1 ) Ran (F 2 ). Conversely for any univariate distribution functions F 1 and F 2 and any copula C, the function F is a two-dimensional distribution function with marginals F 1 and F 2, given by (1). D-B. POUGAZA & al. () MAXIMUM & COPULA MaxEnt 2010 6 / 21

OVERVIEW ABOUT COPULAS IN STATISTICS DIRECT INVERSION METHOD f(x, y) = f 1 (x)f 2 (y)c(f 1 (x), F 2 (y)) c(u, v) = f [ F1 1 (u), F2 1 (v) ] [ f 1 F 1 1 (u) ] [ f 2 F 1 2 (v) ] D-B. POUGAZA & al. () MAXIMUM & COPULA MaxEnt 2010 7 / 21

OVERVIEW ABOUT COPULAS IN STATISTICS ELLIPTICAL COPULAS 1 Gaussian Copula : with one parameter ρ Φ Σ : cdf of bivariate standard Gaussian. Σ : covariance matrix, with correlation coefficientρ C ρ (u, v) = Φ Σ ( Φ 1 (u), Φ 1 (v) ) 2 Student Copula with two parameters ρ and ν t Σ,ν : cdf of a bivariate Student distribution, Σ : covariance matrix with correlation coefficient ρ, ν : the degree of freedom ( C ρ,ν (u, v) = t Σ,ν t 1 ν (u), t 1 ν (v) ) D-B. POUGAZA & al. () MAXIMUM & COPULA MaxEnt 2010 8 / 21

OVERVIEW ABOUT COPULAS IN STATISTICS ARCHIMEDEAN COPULAS For ϕ a non increasing and convex function, where ϕ(0) =, ϕ(1) = 0 : C(u, v) = ϕ 1 (ϕ(u) + ϕ(v)). Important : Multivariate dependence is captivated by an univariate function. Some examples : 1 Clayton Copula (1978) : with one non-null parameter α [ 1, ) ( t α 1 ) The generator ϕ(t) = 1 α C(u, v; α) = [ [u α + v α 1 ] 1 ] α 2 Gumbel Copula (1960) : with one parameter 0 < α 1. The generator ϕ(t) = ln(1 α ln(t)) C α (u, v) = uv exp ( α ln u ln v). + D-B. POUGAZA & al. () MAXIMUM & COPULA MaxEnt 2010 9 / 21

MAXIMUM ENTROPIES COPULAS Maximum Entropies Copulas Problem : Given the two marginals f 1 (x) and f 2 (y) find the joint pdf f(x, y) Solution : Select the solution which maximizes an entropy Mathematics : Maximize J 1 (f) = f(x, y) ln f(x, y) dx dy, subject to C 1 : f(x, y) dy = f 1 (x), C 2 : f(x, y) dx = f 2 (y), C 3 : f(x, y) dx dy = 1. x y D-B. POUGAZA & al. () MAXIMUM & COPULA MaxEnt 2010 10 / 21

MAXIMUM ENTROPIES COPULAS Differents Entropies Expressions Rényi, Burg and Tsallis entropies J 2 (f) = 1 ( ) 1 q ln f q (x, y) dx dy, q 0 and q 1, J 3 (f) = ln f(x, y) dx dy, J 4 (f) = 1 ( ) 1 f q (x, y) dx dy q 0 and q 1. 1 q D-B. POUGAZA & al. () MAXIMUM & COPULA MaxEnt 2010 11 / 21

MAXIMUM ENTROPIES COPULAS Lagrange Multipliers Technique L(f, λ 0, λ 1, λ 2 ) = J i (f) + λ 0 (1 ( + λ 1 (x) f 1 (x) + λ 2 (y) ( f 2 (y) ) f(x, y)dxdy ) f(x, y)dy dx ) f(x, y)dx dy, Solution L/ f = 0 L/ λ 0 = 0 L/ λ 1 = 0 L/ λ 2 = 0. D-B. POUGAZA & al. () MAXIMUM & COPULA MaxEnt 2010 12 / 21

MAXIMUM ENTROPIES COPULAS Shannon s Entropy F (x 1,..., x n ) = f(x, y) = f 1 (x)f 2 (y) n f(x 1,..., x n ) = f i (x i ) x1 0 xn... 0 i=1 i=1 n n f i (s i ) ds i, 0 x i 1. i=1 D-B. POUGAZA & al. () MAXIMUM & COPULA MaxEnt 2010 13 / 21

MAXIMUM ENTROPIES COPULAS Tsallis entropy index q = 2 Pdf : Cdf : f(x, y) = [f 1 (x) + f 2 (y) 1] +. [ n ] f(x 1,..., x n ) = f i (x i ) n + 1 i=1 + F (x, y) = x y 0 0 f(s, t) ds dt F (x, y) = y F 1 (x) + x F 2 (y) x y, 0 x, y 1. F (x 1,..., x n ) = n n n F i (x i ) x j + (1 n) x i, 0 x i 1. i=1 j=1 j i i=1 D-B. POUGAZA & al. () MAXIMUM & COPULA MaxEnt 2010 14 / 21

NEW FAMILIES OF COPULAS Direct Inversion Method Given the expressions of F (x, y) and F 1 (x), F 2 (y), the expressions of copula becomes : C(u, v) = [ u F 1 2 (v) + v F 1 1 (u) F 1 1 (u) F 1 2 (v) ] + Multivariate case : n C(u 1,..., u n ) = i=1 u i n F 1 j=1 j i j (u j ) + (1 n) n i=1 i (u i ) F 1 + D-B. POUGAZA & al. () MAXIMUM & COPULA MaxEnt 2010 15 / 21

NEW FAMILIES OF COPULAS Beta Distribution Marginals One great family of distributions defined on [0, 1] : where f 1 (x) = f 2 (y) = 1 B(a 1, b 1 ) xa1 1 (1 x) b1 1 1 B(a 2, b 2 ) ya2 1 (1 y) b2 1, B(a i, b j ) = 1 Interesting particular cases : case 1 : a i > 0, b j = 1 case 2 : a i = b j = 1/2 0 t ai 1 (1 t) bj 1 dt, 0 x, y 1 and a i, b j > 0. D-B. POUGAZA & al. () MAXIMUM & COPULA MaxEnt 2010 16 / 21

NEW FAMILIES OF COPULAS New Families of copulas case 1 : a i > 0, b j = 1 { f 1 (x) = a 1 x a1 1 F 1 (x) = x a1 F1 1 (u) = u 1 a 1 f 2 (y) = a 2 y a2 1 F 2 (y) = y a2 F2 1 (v) = v 1 a 2 The corresponding copula : F (x, y; a 1, a 2 ) = y x a1 + x y a2 x y, 0 x, y 1. } C(u, v; a 1, a 2 ) = u v 1 a 2 + v u 1 a 1 u 1 a 1 v 1 a 2 is well defined for appropriate values of a 1, a 2 and for almost u, v in [0, 1]. If a 1 = a 2 = 1 a, C(u, v; a) = (u v) a (u 1 a 1 v 1 a ) where u a v = [u a + v a 1] 1 a is the generalized product. D-B. POUGAZA & al. () MAXIMUM & COPULA MaxEnt 2010 17 / 21

NEW FAMILIES OF COPULAS New Families of copulas Case 2 : a i = b j = 1/2 1 f 1 (x) = π x(1 x) F 1(x) = 2 π arcsin( x) F1 1 (u) = sin 2 ( π 2 u) 1 f 2 (y) = π y(1 y) F 2(y) = 2 π arcsin( y) F2 1 (v) = sin 2 ( π 2 v) F (x, y) = 2y π arcsin( x) + 2x π arcsin( y) x y, 0 x, y 1. The corresponding copula : C(u, v) = u sin 2 ( πv 2 ) + v sin2 ( πu 2 ) sin2 ( πu 2 ) sin2 ( πv 2 ) is well defined for all u, v in [0, 1]. D-B. POUGAZA & al. () MAXIMUM & COPULA MaxEnt 2010 18 / 21

CONCLUSION New families of copula, other examples are in investigation Link betwen copula & Tomography Shannon : f(x, y) = f 1 (x)f 2 (y) : Multiplicative Backprojection Tsallis : f(x, y) = [f 1 (x) + f 2 (y) 1] + : Backprojection Method D-B. POUGAZA & al. () MAXIMUM & COPULA MaxEnt 2010 19 / 21

Thanks to Dr. Durante Fabrizio Pr. Christian Genest Dr. Jean-François Bercher Pr. Christophe Vignat D-B. POUGAZA & al. () MAXIMUM & COPULA MaxEnt 2010 20 / 21

BIBLIOGRAPHY Link between Copula and Tomography Pattern Recognition Letters, Elsevier, 2010. D-B. Pougaza, A. Mohammad-Djafari & J-F. Bercher Utilisation de la notion de copule en tomographie GRETSI 2009, Dijon France. D-B. Pougaza, A. Mohammad-Djafari & J-F. Bercher Copula and Tomography VISSAP 2009, Lisbon, Portugal. A. Mohammad-Djafari & D-B. Pougaza Fonctions de répartition à n dimensions et leurs marges Publications de l Institut de Statistique de L Université de Paris 8, 1959 Abe Sklar D-B. POUGAZA & al. () MAXIMUM & COPULA MaxEnt 2010 21 / 21