Concentration, self-bounding functions

Transcription

1 Concentration, self-bounding functions S. Boucheron 1 and G. Lugosi 2 and P. Massart 3 1 Laboratoire de Probabilités et Modèles Aléatoires Université Paris-Diderot 2 Economics University Pompeu Fabra 3 Département de Mathématiques Université Paris-Sud Cachan, 03/02/2011 Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

2 Context Motivations X 1,..., X n : X-valued, independent random variables. F : X n R Z = F(X 1,..., X n ) Goal : upper-bounds on log E [ λ(z EZ e )] t > 0 P {Z EZ + t} and P {Z EZ t} Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

3 Motivations Context... Non-asymptotic tail bounds for functions of many independent random variables that do not depend too much on any of them. 1 high dimensional geometry ; 2 random combinatorics ; 3 statistics ; A variety of methods 1 Martingales 2 Talagrand s induction method 3 Transportation method 4 Entropy method 5 Chatterjee s method (exchangeable pairs) Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

4 Motivations Inspiration: Gaussian concentration Theorem (Tsirelson, Borell, Gross,..., 1975) X 1,..., X n i.i.d. N(0, 1) F : R n R L-Lipschitz (w.r.t.) Euclidean distance Z = F(X 1,..., X n ) var[z ] L 2 Poincaré s inequality log E [ e λ(z EZ )] λ2 L 2 P {Z EZ + t} e t2 /(2L 2 ) 2 Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

5 Motivations Efron-Stein inequalities (1981) Z = F(X 1, X 2,..., X n ), (independent R.V) X 1,..., X n X 1,..., X n but from X 1,..., X n. For each i {1,..., n} Z = F(X i 1,..., X i 1, X i, X i+1,...x n ). X (i) = (X 1,..., X i 1, X i+1,..., X n ). F i : a function of n 1 arguments Z i = F i (X 1,..., X i 1, X i+1, X n ) = F i (X (i) ). Theorem (Jackknife estimates of variance are biased) V + = n E [ (Z Z i )2 + X ] 1,..., X n V = i (Z Z i ) 2 Var[Z ] E[V + ] E[V ] Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

6 Motivations Exponential Efron-Stein inequalities Theorem (Sub-Gaussian behavior) If V + v then, for λ 0 log E [ e λ(z EZ )]] λ2 v 2 Theorem (B., Lugosi and Massart, 2003) For 0 λ 1/θ, log E [ e λ(z EZ ]] λθ (1 λθ) log E [ e ] λv +/θ Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

7 Motivations Entropy method Entropy Y an X-valued random variable f non-negative (measurable) function over X Ent [f ] = E [f (Y ) log f (Y )] E[f (Y )] log E[f (Y )]. Why? if Y = exp(λ(z EZ )), let G(λ) = 1 λ log E [ eλ(z EZ )] [ 1 Ent ] e λ(z EZ ) λ 2 E [ eλ(z EZ )] = dg(λ) dλ Basis of Herbst s argument : bounds on Entropy can be translated into differential inequalities for logarithmic moment generating functions. Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

8 Motivations Gross logarithmic Sobolev inequality Theorem (Gross,..., 1975) X 1,..., X n i.i.d. N(0, 1) F : R n R differentiable Z = F(X 1,..., X n ) [ F(X1 var[z ] E,..., X n ) ] 2 Ent [ Z 2] 2E [ F 2] Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

9 Motivations Bounds on entropy Subadditivity X 1,..., X n random variables. Z = f (X 1,..., X n ) 0 Ent (i) [Z ] = E (i) [Z log Z ] E (i) Z log E (i) Z Ent [f (X 1,..., X n )] n E [ Ent (i) [Z ] ] Upper-bounding Entropy of a function of a single random variable Expected value minimizes expected Bregman divergence with respect to convex function x x log x Ent [Z ] inf E [Z (log Z u) (Z u)] u>0 Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

10 Motivations Entropy method in a nutshell Summary The entropy method converts a modified logarithmic Sobolev inequality into a differential inequality involving the logarithm of the moment generating function of Z. Starting point Theorem (a modified logarithmic sobolev inequality.) let φ(x) = e x x 1. For any λ R, λe [ Ze λz ] E [ e λz ] log E [ e λz ] n E [ e λz φ ( λ(z Z i )) ]. Use different conditions to upper-bound n φ ( λ(z Z i))... Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

11 Square roots Variance stabilization Folklore Z 0 and Var[Z ] aez Var[ Z ] a If Z n Pois(nµ), then Z n E Z n N(0, 1/4) (Cramer s Delta) Lemma If X Pois, let v = (EX)E[1/(4X + 1)], for λ 0, [ log E e λ( X E[ ] X]) vλ(e λ 1). for t > 0 P { X E X + t } exp ( t2 ( log 1 + t )). 2v Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

12 Proof Square roots Variance stabilization Poisson Poincaré inequality (Klaassen 1985) X Pois, Z = f (X) Var[Z ] EX E[ Df (X) 2 ], with Df (X) = f (X + 1) f (X). Consequence: Var[ X] v. Poisson logarithmic Sobolev inequality (L. Wu, Bobkov-Ledoux) Ent[Z ] EX E [Df D log f ] Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

13 Self-bounding Self-bounding property (I) Flavors of self-bounding f : X n R is said to have the self-bounding property if f i : X n 1 R : 1 x = (x 1,..., x n ) X n and i = 1,..., n, 0 f (x) f i (x (i) ) 1 2 n ( f (x) fi (x (i) ) ) f (x). where x (i) = (x 1,..., x i 1, x i+1,..., x n ). Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

14 Self-bounding Flavors of self-bounding Examples 1 Suprema of positive bounded empirical processes. X i = (X i,s ) s T, T finite, 0 X i,s 1. X i independent. Z = sup s T 2 Suprema of bounded empirical processes X i,s 1 (relaxing the second assumption) n 3 Largest eigenvalue of a Gram matrix sup u T u : u 2 =1 n X i,s X i X T i u Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

15 Self-bounding Concentration inequalities Binomial-Poisson tails If T = 1, Bennet inequality holds h(u) = (1 + u) log(1 + u) u, and u 1 φ(v) = sup (uv h(u)) = e v v 1. u 1 log E [ e λ(z EZ )] φ(λ)ez λ R. ( ( )) t P {Z EZ + t} exp EZh EZ t > 0 Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

16 Chi-square tails Self-bounding Concentration inequalities X i µ i χ 2 weighted chi-square random variables 1 Z = n X i v = 2 n µ2 and c = max i i µ i /2 Bernstein inequality log E [ e λ(z EZ )] vλ 2 2(1 cλ) ( t 2 ) P {Z EZ + t} exp 2(v + ct) Bennett inequality entails Bernstein inequality (with scale factor 1/3) Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

17 Self-bounding Concentration inequalities Self-bounding property and concentrations inequalities h(u) = (1 + u) log(1 + u) u, and u 1 φ(v) = sup (uv h(u)) = e v v 1. u 1 Theorem (B., Lugosi and Massart ) If Z satisfies the self-bounding property, log E [ e λ(z EZ )] φ(λ)ez λ R. ( ( t P {Z EZ + t} exp EZh EZ ( ( )) t P {Z EZ t} exp EZh EZ )) t > 0 0 < t EZ. Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

18 ... with applications Self-bounding Concentration inequalities Definition (Conditional Rademacher averages) ɛ 1,..., ɛ n Rademacher variables (x 1,..., x n ) F(x 1,..., x n ) = E Symmetrization inequalities (Giné and Zinn, 1984) 1 2 E [F(X 1,..., X n )] E sup s T sup s T n ɛ i x i,s n X i,s 2E [F(X 1,..., X n )] Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

19 Self-bounding Concentration inequalities Theorem (B., Lugosi and Massart 2003) Conditional Rademacher averages are self-bounding. Consequences while Var[sup s T Var[F(X 1,..., X n )] E[F(X 1,..., X n )] n X i,s ] sup s T n Var[X 1,s ] + 2E[sup s T n X i,s ]. Conditional Rademacher averages : kind of weighted Bootstrap estimates. Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

20 Variations on a theme Self-bounding Concentration inequalities Definition (weakly (a, b)-self-bounding) f : X n [0, ) is weakly (a, b)-self-bounding if f i : X n 1 [0, ) : x X n n ( f (x) fi (x (i) ) ) 2 af (x) + b. Definition (strongly (a, b)-self-bounding) f : X n [0, ) is strongly (a, b)-self-bounding if f i : X n 1 [0, ) : i = 1,..., n, andx X n, 0 f (x) f i (x (i) ) 1, and n ( f (x) fi (x (i) ) ) af (x) + b. Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

21 Self-bounding Concentration inequalities Definition (Submodular function) f : 2 n R f (A B) + f (A B) f (A) + f (B) Submodularity implies neither monotonicity nor non-negativity Example The capacity of cuts in a directed graph is submodular. Lemma (Vondrák) Non-negative 1-Lipschitz submodular function are (2, 0)-self-bounding. Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

22 Self-bounding Concentration inequalities Efron-Stein inequality n Var[Z ] E (f (X) f i (X (i) )) 2 Remark Both definitions imply that Z = f (X) satisfies Var[Z ] aez + b. Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

23 Self-bounding Concentration inequalities Theorem (Maurer 2006) X = (X 1,..., X n ) X-valued independent random variables. f : X n [0, ) weakly (a, b)-self-bounding function (a, b 0). Let Z = f (X). If i n, x X n, f i (x (i) ) f (x), then for all 0 λ 2/a, log E [ e λ(z EZ )] (aez + b)λ2 2(1 aλ/2) and for all t > 0, ( t 2 ) P {Z EZ + t} exp 2 (aez + b + at/2). Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

24 Lower tails Self-bounding Concentration inequalities Theorem (McDiarmid and Reed 2008, B., Lugosi, Massart, 2009) X = (X 1,..., X n ) X-valued independent random variables. Let f : X n [0, ) be a weakly (a, b)-self-bounding function (a, b 0). Let Z = f (X) and define c = (3a 1)/6. If, f (x) f i (x (i) ) 1 for each i n and x X n, then for 0 < t EZ, ( t 2 ) P {Z EZ t} exp. 2 (aez + b + c t) If a 1/3, sub-gaussian behavior. Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

25 Upper tails Self-bounding Concentration inequalities Theorem (McDiarmid and Reed 2008, B., Lugosi, Massart, 2009) X = (X 1,..., X n ) X-valued independent random variables. Let f : X n [0, ) be a weakly (a, b)-self-bounding function (a, b 0). Let Z = f (X) and define c = (3a 1)/6. Then for all λ 0, log E [ e λ(z EZ )] (aez + b)λ2 2(1 c + λ) and for all t > 0, ( t 2 ) P {Z EZ + t} exp 2 (aez + b + c + t). If a 1/3, sub-gaussian behavior. Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

26 Self-bounding Concentration inequalities Proofs Remark The entropy method converts a modified logarithmic Sobolev inequality into a differential inequality involving the logarithm of the moment generating function of Z. Starting point Theorem (a modified logarithmic sobolev inequality.) For any λ R, λe [ Ze λz ] E [ e λz ] log E [ e λz ] n E [ e λz φ ( λ(z Z i )) ]. Use different conditions to upper-bound n φ ( λ(z Z i))... Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

27 Proofs (...) Self-bounding Concentration inequalities Establishing differential inequalities for G(λ) = log E [ eλ(z EZ )] 1 (a, b)-self-bounding: Ent [ e λz ] φ( λ)e [ (az + b) e λz ] 2 (a, b)-weakly self-bounding, λ 0: Ent [ e ] λz λ2 2 E [ (az + b)e ] λz. 3 (a, b)-weakly self-bounding, λ 0: Ent [ e λz ] φ( λ)e [ (az + b)e λz ]. Key differential inequality where v = aez + b. [λ aφ ( λ)] G (λ) G(λ) vφ( λ), (1) Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

28 Self-bounding Around the Herbst argument Concentration inequalities Lemma Let f : I R,, C 1 where interval I 0 with f (0) = 0. Assume x 0 f (x) 0. Let g be C 0 on I and G be C on I with G(0) = G (0) = 0 and for every λ I, f (λ)g (λ) f (λ)g(λ) f 2 (λ)g(λ). Then, for every λ I, G(λ) f (λ) λ 0 g(x)dx. Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

29 Comparisons Self-bounding Concentration inequalities Let ρ be C 0 on I 0. Let a 0. Let H : I R, be C satisfying λh (λ) H(λ) ρ(λ) (ah (λ) + 1) with ah (λ) + 1 > 0 λ I and H (0) = H(0) = 0. Let ρ 0 : I R, assume that G 0 : I R is C with λ I, ag 0 (λ) + 1 > 0 and G 0 (0) = G 0(0) = 0 and G 0 (0) = 1. Assume also that G 0 solves the differential equation λg 0 (λ) G 0(λ) = ρ 0 (λ) ( ag 0 (λ) + 1). If ρ(λ) ρ 0 (λ) for every λ I, then H G 0. Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

30 Sketch of proof Self-bounding Concentration inequalities Key differential inequality λg (λ) G(λ) φ( λ)(1 + ag (λ))) 1 2G γ (λ) = λ 2 /(1 γλ) solves λh (λ) H(λ) λ 2 (1 + γh (λ))) 2 Choosing γ = a works for λ 0 3 May not be the best choice for ρ γ... 4 Optimizing ρ γ = (λg γ(λ) G γ (λ))/(1 + ag γ(λ))) leads to the desired result Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

31 Talagrand s convex distance Application: Convex distance Definition and motivations Definition A X n B 2 n unit ball in Rn endowed with euclidean metric Theorem d T (X, A) = inf y A sup α B n 2 n α i I Xi y i M(A) : Probability distributions supported by A d T (X, A) = sup α B n 2 inf ν M(A) n α i ν{x i Y i } Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

32 Modus operandi Talagrand s convex distance Definition and motivations Lemma For any A X n and x X n, the function f (x) = d T (x, A) 2 satisfies where f i is defined by 0 f (x) f i (x (i) ) 1 f i (x (i) ) = inf f (x 1,..., x i 1, x x i X i, x i+1,..., x n ). (2) Moreover, f is weakly (4, 0)-self-bounding. Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

33 Talagrand s convex distance Definition and motivations Efron-Stein estimates of the variance of d T Efron-Stein estimate of variance of d T (, A) n ( V + = f (x) f i (x (i) ) Lemma For all A X n, V + is bounded by 1. V + 1. Var [d T (X, A)] 1. ) 2 Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

34 Talagrand s convex distance Definition and motivations A consequence of the minmax characterization of d T 1 M(A) : the set of probability measures on A. 2 we may re-write d T as d T (x, A) = inf sup ν M(A) α: α 2 1 j=1 where Y = (Y 1,..., Y n ) is distributed according to ν. 3 By the Cauchy-Schwarz inequality, d T (x, A) 2 = inf ν M(A) j=1 n α j E ν [I xj Y j ] (3) n ( Eν [I xj Y j ] ) 2. Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

35 Talagrand s convex distance Definition and motivations Weak self-boundedness of d 2 T Denote the pair (ν, α) at which the saddle point is achieved by ( ν, α). For all x, ( n f 2 ( f 2 (x) f i (x )) (i) 1 since (x) f i (x )) (i) α 2. i n ( f (x) fi (x (i) ) ) 2 = n n ( ) 2 ( ) 2 f (x) f i (x (i) ) f (x) + f i (x (i) ) α 2 i 4f (x). 4f (x) Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

36 Talagrand s convex distance Self-bounding : example Talagrand s convex distance inequality Theorem (Talagrand 1995) ] P{A}E [e d 2 T (X,A)/4 1 Proof. P{X A} = P { d T (X, A) 2 E [ d 2 T (X, A)] t } ( exp E[d T (X, A) 2 ) ]. 8 For 0 λ 1/2, log E [ e λ(z EZ )] λ2 2EZ 1 2λ. Choosing λ = 1/10 leads to the desired result. Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

37 Suprema of non-centered empirical processes Talagrand-Bousquet Suprema of non-centered empirical processes Well-understood scenarios 1 Suprema of positive bounded empirical processes: self-bounding property 2 Suprema of centered bounded empirical processes : Talagrand s inequality (revisited by Ledoux, Massart, Rio, Klein, Bousquet,...). Bennett inequality with variance factor coinciding with the Efron-Stein estimate of variance Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

38 Suprema of non-centered empirical processes Talagrand-Bousquet Talagrand-... -Bousquet inequality Z = sup s T n X i,s X 1,..., X n identifically distributed EX i,s = 0 and 1 X i,s 1 σ 2 = sup s T n var[x i,s] Efron-Stein estimate of variance var[z ] v = 2EZ + σ 2 log E [ e λ(z EZ )] vφ(λ) For x > 0 { P Z EZ + 2vx + x } e x 3 Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

39 Suprema of non-centered empirical processes Empirical excess risk Another scenario : Excess Empirical Risk ŝ, s R(s) = E[X i,s ] Risk of s T R n (s) = 1 n n X i,s s : R( s) = EX i, s = inf s T EX i,s = inf s T R(s) ŝ : nr n (ŝ) = n X i,ŝ = inf s T n X i,s = inf s T R n (s) Excess risk and empirical counterpart Excess risk R(ŝ) R( s) Excess empirical risk (EER) Z = n(r n ( s) R n (ŝ)) = sup s T n (X i, s X i,s ) = n ( ) Xi, s X i,ŝ Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

40 Suprema of non-centered empirical processes Variance bounds for EER Empirical excess risk Consequences of Efron-Stein inequalities : Var [Z ] 2 E n 1 (X i, s EX i, s ) (X i,ŝ EX i,ŝ ) and Var [Z ] n n 2 E (X i, s X i,ŝ ) 2 + E (X i, s X + E i,ŝ )2 n (X i, s X i,ŝ ) 2 Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

41 Suprema of non-centered empirical processes Empirical excess risk Consequences of Talagrand s inequalities (and peeling) Assumptions d a distance over T, ψ, ω : [0, 1] R +,, ψ(x)/x, ω(x)/x : ne sup s : d(s, s) r (R(s) R n (s)) (R n ( s) R( s)) ψ(r) E [ (X i,s X i, s ) 2] ( ) 2 d(s, s) 2 ω R(s) R( s) Definition r is the positive solution of nr 2 = ψ(ω(r)) Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

42 Suprema of non-centered empirical processes Empirical excess risk Consequences, cont d With probability larger than 1 δ max (R(ŝ) R( s), R n ( s) R(ŝ)) κ (r 2 + ω(r ) 2 nr 2 log 1 ) δ r 2 is called the rate of the estimation problem max (E [R(ŝ) R( s)], E [R n ( s) R(ŝ)]) κ r 2 Combining... var [Z ] = var [ n(r n ( s) R n (ŝ)) ] nκ ( ω(r ) 2) Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

43 Suprema of non-centered empirical processes Bernstein inequality for EER Empirical excess risk Theorem (B., Bousquet, Lugosi, Massart, 2005) (Z EZ ) q + 3q V + q Bernstein like inequality ( n(rn ( s) R n (ŝ)) ) ( q nqω(r κ ) + ( ) ) ω(r ) nω q nr Variance factor nω(r ) 2 Scale factor ( ) ω(r ) nω nr Works for some statistical learning problems (learning VC-classes under good noise conditions). Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44

44 References References 1 B. and Massart : A high dimensional Wilks phenomenon. Probability Theory and Related Fields online. (2010) 2 B., Lugosi and Massart : On concentration of self-bounding functions. Electronic Journal of Probability 14 (2009) and 1 Maurer. Concentration inequalities for functions of independent variables, Random Structures and Algorithms, 29 (2006) McDiarmid and Reed. Concentration for self-bounding functions and an inequality of talagrand. Random Structures and Algorithms, 29 (2006) B. Lugosi and Massart. A sharp concentration inequality with applications. Random Structures and Algorithms, 16 (2000), Boucheron & Lugosi & Massart (LPMA et al.) Concentration, self-bounding functions Cachan, 03/02/ / 44