Next, we show that, for any x Ø, (x) Æ 3 x x HW3 Problem i) We first show that for a random variable X bounded in [, ], since x apple x, wehave Var[X] EX [EX] apple EX [EX] EX( EX) Since EX [, ], therefore, Var(X) apple EX( EX) apple with equality holds when EX For any random variable Y bounded in [a, b], X Y a is a random variable bounded in [, ] So b a Var[Y ](b a) Var[X] (b a) EX( EX) apple (b a), with equality holds when EY b+a ii) The equality is tight Define a random variable X as ( P(X b) P(X a) Then Var[X] (b a) Question (i) Let Gamma Function : (, Œ) be defined by (x) t x e t dt It iw well known that (x + ) x (x) for any x> (By integral by parts) The second derivative d (x) d is x d (x) d x t x (ln t) e t dt > This implies that (x) is convex on (, Œ) In particular, (x) is convex on [, ], hence, for any x œ [, ], (x)! ( x)+ (x ) " Æ ( x) () + (x ) () Moreover, this, together with the formula (x + ) x (x) for any x>, yields that, for any x œ (, ), x (x) (x + ) Æ Æ x Æ By property, one has (x) Æ x Æ On the other hand,
Æ Æ On the other hand, d ln(x x ) +ln(x) Ø ln Ø dx Ë È This implies that x x is increasing on,,thus, / 3 x x Ø 3 Ø Hence (x) Æ 3 x x x> Let [x] be the largest integer such that [x] Æ x Thus,x [x]+x [x] and Æ x [x] < or Æ x [x]+< By property, one gets (x) ([x]+x [x]) (x ) (x ) (x [x] + ) (x [x] + ) Æ x x x Æ x [x] Æ 3 x x [x] Let x p, one gets, for any p Ø, (ii) Let f :[, Œ) æ R be a function defined by p Æ 3 p p f(x) x x x Ø Taking derivatives of lnf, one gets d lnf dx ln(x) x Æ x Æ e ln(x) Ø, thenlnf and f is increasing on (,e] x Ø e ln(x) Æ, thenlnf and f is decreasing on [e, Œ) Thus, f attains the maximum at x e Hence, for p Ø, one gets Note that e Æ 3 Æ ( Ô ) Æ ( Ô ) e One gets p p Æ e /e p p Æ e /e Æ Ô Question 3 In the follwowing, we will show four pairs of equivalences, ie, (b) (c), (b) (d), (c) (d) and (x) (c) First of all, we prove ( ) ( )
(b) (c) Let f(x) be the pdf of X Hence, for p Ø, one has E{ X p } Æ Œ Œ This implies that /p E{ X p } Æ 3 /p p /p K / where K 3 Ô K / Ô x p f(x) dx 3 x 3 p K p/ p K p/ p t p dt f(x) dx f(x)dx p t p dt x >t p t p P( X >t) dt p t p e t /K dt Æ 3 p K p/ u p/ e u du p p p/ Ô Ôp Æ 3 Ô K / (c) (b) By Taylor expansion, for any n Ø, one has Œ e n n k k! Ø nn k For any n Ø and t>, by Markov inequality, one gets Ô Ôp K Ôp, P( X >t)p( X n >t n ) Æ E{ X n } t n Æ Kn ( Ô n) n t n ( K ) n n n t n Let a e K and K e K, thus one has Œ (a t ) n e a t P( X >t) P( X >t) n Œ (a t ) n P( X >t)+ P( X >t) n Œ (a t ) n Æ + ( K ) n n n t n n Œ + ( a K) n nn n Œ Æ + ( a K) n e n n + e a K e a K
(d) (c) Note that, for any a>, ; Œ (a X ) n < Œ ; (a X ) n < Œ E{e a X } E E Thus, for n Ø and a /K 3, This implies and n n K3 n E{Xn } Æ E{e X /K 3 } Æ E{X n } Æ K n 3 Æ K n 3 ( n) n E{X n } /n Æ apple K 3 Ô n n a n E{Xn } Note that E( X p ) /p is increasing with respect to p Ø (by Holder inequality) Thus, for n Ø, E{ X n } /(n ) Æ E{ X n } /n Æ apple K 3 Ô n apple K 3 Ôn Ô n Ô n Since lim næœ Ô I n Ô,thus n Ô J Œ n Ô is bounded, say by K ÕÕ > Hence, n n E{ X n } /(n ) Æ apple K 3 Ôn Ô n Ô n Æ apple K 3 K ÕÕ Ôn Let K Ô K 3 K ÕÕ and one gets E( X p ) /p Æ K Ôp In the following, we will prove (a) (c) Let Y X µ, whereµ E(X) Clearly, E(Y )and thus E(e (X µ) ) Æ e /K E(e Y ) Æ e /K E( Y p ) /p Æ K Õ Ôp E( X µ p ) /p Æ K Õ Ôp for p Ø Thus, in order to prove (a) (c), itsu ces to show that E( X µ p ) /p Æ K Õ Ôp for p Ø E( X p ) /p Æ K Ôp for p Ø Let d f(x)dx, thus,e( X µ p ) /p ÎX µî p, and E( X p ) /p ÎXÎ p, (c) (a) Thus, by triangle inequality, one has E( X µ p ) /p ÎX µî p, ÆÎXÎ p, + εΠp, ÎXÎ p, + µ Æ K Ô p + µ Note that when p Ø µ +, Ô p Ø µ ; and when Æ p< µ +, there exists a constant K Õ > such that µ Æ K Õ Ôp, since there are only finite terms of p Hence, where K Õ K + K Õ E( X µ p ) /p Æ K Ô p + µ Æ K Õ Ôp, (a) (c) Similarly, by triangle inequality, one gets E( X p ) /p ÎXÎ p, ÆÎX µî p, + εΠp, ÎX µî p, + µ Æ K Õ Ô p + µ Æ K Ôp, where K K Õ + K Õ
Question Let S be a subset of {,,,p}, S be the size of S, X be a n p matrix and X T X/n Compatibility Condition: there exist some constants v > and > such that b j Æ S (b T b), v b j Æ b j Restricted Eigenvalue Condition: there exist some constants v > and > such that Æ b T b v b j b j Æ b j Restricted Eigenvalue Condition Compatibility Condition Let v be such that Æ b T b b j Æ b j v b j Thus, by Cauchy Schwartz inequality, one has b j v b j Æ v v b j v S b j Æ S (b T b) b j Æ b j
Question 5 Let Y œ R n, X be a n p matrix For Ø fixed, let ˆ arg min œr p where ÎxÎ n ÎxÎ /n for any x œ R n and Î Î Thus, for any œ R p, one has Y X n + Î Î, p j for any œ R p Y X ˆ n + Î ˆ Î Æ Y X n + Î Î Let t œ [, ] Replacing in the rightside by ˆ + t ( ˆ ), one gets Y X ˆ + Î ˆ Î n Æ Y X ˆ t X( ˆ ) + Î ˆ + t ( ˆ )Î n, j and then Î ˆ Î Æ t X( ˆ ) n t ÈY X ˆ,X( ˆ )Í n + Î ˆ + t ( ˆ )Î BY convexity of the L norm, one gets or Î ˆ Î Æ t Æ t X( ˆ ) n t ÈY X ˆ,X( ˆ )Í n + ( t) Î ˆ Î + t Î Î X( ˆ ) n t ÈY X ˆ,X( ˆ )Í n t Î ˆ Î + t Î Î Divididing t from both sides and letting t æ +, one has Æ ÈY X ˆ,X( ˆ )Í n + Î Î Î ˆ Î Note that Thus, ÈY X ˆ,X( ˆ )Í n X( ˆ ) n ÈY X,X( ˆ )Í n Æ X( ˆ ) n + ÈY X,X( ˆ )Í n + Î Î Î ˆ Î and X( ˆ ) n + Î ˆ Î ÆÈY X,X( ˆ )Í n + Î Î