D KL (P Q) := p i ln p i q i
|
|
- Dana Wood
- 6 years ago
- Views:
Transcription
1 Cheroff-Bouds 1 The Geeral Boud Let P 1,, m ) ad Q q 1,, q m ) be two dstrbutos o m elemets, e,, q 0, for 1,, m, ad m 1 m 1 q 1 The Kullback-Lebler dvergece or relatve etroy of P ad Q s defed as m D KL P Q) : l q If m 2, e, P, 1 ) ad Q q, 1 q), we also wrte D KL q) The Kullback-Lebler dvergece rovdes a measure of dstace betwee the dstrbutos P ad Q: t reresets the exected loss of effcecy we cur f we ecode a m-letter alhabet wth dstrbuto P wth a code that s otmal for dstrbuto Q We ca ow state the geeral form of the Cheroff-Boud: Theorem 11 Let X 1,, X be deedet radom varables wth X {0, 1} ad Pr[X 1, for 1, Set X : 1 X The, for ay t [0, 1, we have 1 Pr[X + t) e DKL+t ) 2 Three Proofs 21 The Momet Method The usual roof of Theorem 11 uses the exoetal fucto ex ad Markov s equalty It s called momet method because ex smultaeously ecodes all momets of X, e, X, X 2, X 3, etc The roof techque s very geeral ad ca be used to obta several varats of Theorem 11 Let λ > 0 a arameter to be determed later We have From Markov s equalty, we obta Pr[X + t) Pr[λX λ + t) Pr [ e λx e λ+t) Pr [ e λx e λ+t) E[eλX e λ+t) Now, the deedece of the X yelds [ E[e λx E [e λ 1 X E e λx 1 1 E [e λx e λ + 1 ) Thus, Pr[X > + t) e λ + 1 ), 1) e λ+t) 1
2 for every λ > 0 Otmzg for λ usg calculus, we get that the rght had sde s mmzed f Pluggg ths to 1), we get as desred Pr[X > + t) e λ 1 ) + t) 1 t) [ ) +t 1 ) 1 t e DKL+t ), + t 1 t 22 Chvátal s Method Let B, ) the radom varable that gves the umber of heads deedet Beroull trals wth success robablty It s well kow that ) Pr[B, ) l l 1 ) l, l for l 0,, Thus, for ay τ 1 ad k, we get Pr[B, ) k k ) 1 ) ) k 0 k 1 1 ) τ }{{} k Thus, usg the Bomal theorem, we obta ) Pr[B, ) k 1 ) τ k τ k If we wrte k + t) ad τ e λ, we ca coclude Pr[B, ) + t) ) 1 ) τ k } {{ } ) 1 ) τ k )τ) 1 ) τ + 1 ) τ k e λ + 1 e λ+t) ) Ths s the same as 1), so we ca comlete the roof of Theorem 11 as Secto The Imaglazzo-Kabaets Method Let λ [0, 1 be a arameter to be chose later Let I {1,, } be a radom dex set obtaed by cludg each elemet {1,, } wth robablty λ We estmate the robablty Pr [ I X 1 two dfferet ways, where the robablty s over the radom choce of X 1,, X ad I O the oe had, usg the uo boud ad deedece, we have [ [ Pr X 1 Pr I S X 1 Pr[I S Pr[X 1 S I S {1,,} S {1,,} λ S 1 λ) S S S {1,,} s0 S ) λ) s 1 λ) s λ + 1 λ), 2) s 2
3 by the Bomal theorem O the other had, by the law of total robablty, [ [ Pr X 1 Pr X 1 X + t) Pr[X + t) I I Now, fx X 1,, X wth X + t) For the fxed choce of X 1 x 1,, X x, the robablty Pr [ I x 1 s exactly the robablty that I avods all the X dces where x 0 Thus, [ Pr x 1 1 λ) X 1 λ) 1 t) I Sce the boud holds uformly for every choce of x 1,, x wth X + t), we get [ Pr X 1 X + t) 1 λ) 1 t), so Combg wth 2), I [ Pr X 1 1 λ) 1 t) Pr[X + t) I Pr[X + t) ) λ + 1 λ 3) 1 λ) 1 t) Usg calculus, we get that the rght had sde s mmzed for λ t/1 ) + t) ote that λ 1 for t 1 ) Pluggg ths to 3), [ ) +t 1 ) 1 t Pr[X > + t) e DKL+t ), + t 1 t as desred 3 Useful Cosequeces 31 The Lower Tal Corollary 31 Let X 1,, X be deedet radom varables wth X {0, 1} ad Pr[X 1, for 1, Set X : 1 X The, for ay t [0,, we have Proof Pr[X t) e DKL t ) Pr[X t) Pr[ X t) Pr[X 1 + t), where X 1 X wth X {0, 1} deedet radom varables such that Pr[X result follows from D KL 1 + t 1 ) D KL t ) 1 1 The 3
4 32 Motwa-Raghava verso Corollary 32 Let X 1,, X be deedet radom varables wth X {0, 1} ad Pr[X 1, for 1, Set X : 1 X ad µ The, for ay δ 0, we have e δ ) µ Pr[X 1 + δ)µ 1 + δ) 1+δ, ad Pr[X 1 δ)µ e δ 1 δ) 1 δ ) µ Proof Settg t δµ/ Theorem 11 yelds [ 1 Pr[X 1 + δ)µ ex 1 + δ) l1 + δ) + ) 1 δ/1 )) δ 1 )/ µ 1 + δ) 1+δ e δ2 /1 )+δ 1 + δ) 1+δ Settg t δµ/ Corollary 31 yelds Pr[X 1 δ)µ ex ) µ e δ ) µ 1 + δ) 1+δ [ 1 1 δ) l1 δ) δ/1 )) δ 1 )/ 1 δ) 1 δ ) µ e δ2 /1 ) δ 1 δ) 1 δ ) µ e δ 1 δ) 1 δ ) µ ) δ l ) + δ l )) 1 δ 1 )) 1 + δ 1 33 Hady Versos Corollary 33 Let X 1,, X be deedet radom varables wth X {0, 1} ad Pr[X 1, for 1, Set X : 1 X ad µ The, for ay δ 0, 1), we have Proof By Corollary 32 Usg the ower seres exaso of l1 δ), we get Thus, as clamed 1 δ) l1 δ) 1 δ) Pr[X 1 δ)µ e δ2 µ/2 e δ ) µ Pr[X 1 δ)µ 1 δ) 1 δ 1 δ δ + δ 1) δ + δ2 /2 2 Pr[X 1 δ)µ e [ δ+δ δ2 /2µ e δ2 µ/2, 4
5 Corollary 34 Let X 1,, X be deedet radom varables wth X {0, 1} ad Pr[X 1, for 1, Set X : 1 X ad µ The, for ay δ 0, we have Pr[X 1 + δ)µ e m{δ2,δ}µ/4 Proof We may assume that 1 + δ) 1 The Theorem 11 gves Defe fδ) : D KL 1 + δ) ) The ad By Taylor s theorem, we have Pr[X 1 + δ) e DKL1+δ) ) f δ) l1 + δ) l1 δ/1 )) f δ) 1 + δ)1 δ) 1 + δ fδ) f0) + δf 0) + δ2 2 f ξ), for some ξ [0, δ Sce f0) f 0) 0, t follows that fδ) δ2 2 f ξ) δ ξ) δ δ) For δ 1, we have δ/1 + δ) 1/2, for δ < 1, we have 1/δ + 1) 1/2 Ths gves for all δ 0 ad the clam follows fδ) m{δ 2, δ}/4, Corollary 35 Let X 1,, X be deedet radom varables wth X {0, 1} ad Pr[X 1, for 1, Set X : 1 X ad µ The, for ay δ > 0, we have Proof Combe Corollares 33 ad 34 Pr[ X µ δµ 2e m{δ2,δ}µ/4 Corollary 36 Let X 1,, X be deedet radom varables wth X {0, 1} ad Pr[X 1, for 1, Set X : 1 X ad µ For t 2eµ, we have Proof By Corollary 32 Pr[X t 2 t e δ ) µ Pr[X 1 + δ)µ 1 + δ) 1+δ ) 1+δ)µ e 1 + δ For δ 2e 1, the deomator the rght had sde s at least 2e, ad the clam follows 4 Geeralzatos We meto a few geeralzatos of the three roof techques for Secto 2 Sce the cosequeces from Secto 3 are based o smle algebrac maulato of the bouds, they same cosequeces also hold for the geeralzed settgs 5
6 41 Hoeffdg-Exteso Theorem 41 Let X 1,, X be deedet radom varables wth X [0, 1 ad E[X Set X : 1 X ad : 1/) 1 The, for ay t [0, 1, we have Pr[X + t) e DKL+t ) Proof The roof geeralzes the momet method Let λ > 0 a arameter to be determed later As before, Markov s equalty yelds Pr [ e λx e λ+t) E[eλX e λ+t) Usg deedece, we get E[e λx E [e λ 1 X 1 E [e λx 4) Now we eed to estmate E [ e λx The fucto z e λz s covex, so e λz 1 z)e 0 λ + ze 1 λ for z [0, 1 Hece, E [ e λx E[1 X + X e λ 1 + e λ Gog back to 4), E[e λx 1 + e λ ) Usg the arthmetc-geometrc mea equalty 1 x 1/) 1 x ), for x 0, ths s From here we cotue as Secto 21 1 E[e λx 1 + e λ ) 42 Hyergeometrc Dstrbuto Chvátals roof geeralzes to the hyergeometrc dstrbuto Theorem 42 Suose we have a ur wth N balls, P of whch are red We radomly draw balls from the ur wthout relacemet Let HN, P, ) deote the umber of red balls the samle Set : P/N The, for ay t [0, 1, we have Proof It s well kow that for l 0,, Pr[HN, P, ) + t) e DKL+t ) Pr[HN, P, ) l Clam 43 For every {0,, }, we have N ) 1 P P l ) N P ) N l ) ) ) N l ) 1, ) 6
7 Proof Cosder the followg radom exermet: take a radom ermutato of the N balls the ur Let S be the sequece of the frst elemets the ermutato Let X be the umber of -subsets of S that cota oly red balls We comute E[X two dfferet ways O the oe had, E[X ) Pr[S cotas red balls N ) 1 P ) N P ) ) 5) O the other had, let I {1,, } wth I The the robablty that all the balls the ostos dexed by I are red s P N P 1 N 1 P + 1 ) P N + 1 N Thus, by learty of exectato E[X ) Together wth 5), the clam follows Clam 44 For every τ 1, we have N ) 1 0 P Proof Usg Clam 43 ad the Bomal theorem twce), as clamed N ) 1 0 P ) ) N P τ Thus, for ay τ 1 ad k, we get as before Pr[HN, P, ) k N ) 1 k P ) ) N P τ 1 + τ 1)) ) 1 N ) P N P 0 ) 1 N ) ) P N P 0 0 ) 1 N P τ 1) 0 0 ) ) N P ) 1 τ 1)) ) τ 1) ) N P ) τ 1)) 1 + τ 1)), N by Clam 44 For here the roof roceeds as Secto 22 ) 1 0 P ) ) ) N P )τ k τ + 1 ) τ k, 43 Geeral Imaglazzo-Kabaets Theorem 45 Let X 1,, X be radom varables wth X 0, 1 Suose there exst [0, 1, 1,,, such that for every dex set I {1,, }, we have Pr[ I X 1 I Set X : 1 X ad : 1/) 1 The, for ay t [0, 1, we have Pr[X + t) e DKL+t ) 7
8 Proof Let λ [0, 1 be a arameter to be chose later Let I {1,, } be a radom dex set obtaed by cludg each elemet {1,, } wth robablty λ As before, we estmate the robablty Pr [ I X 1 two dfferet ways, where the robablty s over the radom choce of X 1,, X ad I Smlarly to before, [ Pr X 1 Pr I [ I X 1 S {1,,} S {1,,} [ Pr I S X 1 S [ Pr[I S Pr S X 1 S {1,,} λ S 1 λ) S 6) We defe deedet radom varables Z 1,, Z as follows: for 1,,, wth robablty 1 λ, we set Z 1, ad wth robablty λ, we set Z By 6), ad usg deedece ad the arthmetcgeometrc mea equalty [ [ Pr X 1 E Z I 1 E[Z The roof of the lower boud remas uchaged ad yelds [ Pr X 1 1 λ) 1 t) Pr[X + t), I 1 S 1 λ + λ) 1 λ + λ) 7) as before Combg wth 7) ad otmzg for λ fshes the roof, see Secto
Pr[X (p + t)n] e D KL(p+t p)n.
Cheroff Bouds Wolfgag Mulzer 1 The Geeral Boud Let P 1,..., m ) ad Q q 1,..., q m ) be two dstrbutos o m elemets,.e.,, q 0, for 1,..., m, ad m 1 m 1 q 1. The Kullback-Lebler dvergece or relatve etroy of
More informationRandom Variables. ECE 313 Probability with Engineering Applications Lecture 8 Professor Ravi K. Iyer University of Illinois
Radom Varables ECE 313 Probablty wth Egeerg Alcatos Lecture 8 Professor Rav K. Iyer Uversty of Illos Iyer - Lecture 8 ECE 313 Fall 013 Today s Tocs Revew o Radom Varables Cumulatve Dstrbuto Fucto (CDF
More information2. Independence and Bernoulli Trials
. Ideedece ad Beroull Trals Ideedece: Evets ad B are deedet f B B. - It s easy to show that, B deedet mles, B;, B are all deedet ars. For examle, ad so that B or B B B B B φ,.e., ad B are deedet evets.,
More informationChain Rules for Entropy
Cha Rules for Etroy The etroy of a collecto of radom varables s the sum of codtoal etroes. Theorem: Let be radom varables havg the mass robablty x x.x. The...... The roof s obtaed by reeatg the alcato
More informationLecture 02: Bounding tail distributions of a random variable
CSCI-B609: A Theorst s Toolkt, Fall 206 Aug 25 Lecture 02: Boudg tal dstrbutos of a radom varable Lecturer: Yua Zhou Scrbe: Yua Xe & Yua Zhou Let us cosder the ubased co flps aga. I.e. let the outcome
More informationå 1 13 Practice Final Examination Solutions - = CS109 Dec 5, 2018
Chrs Pech Fal Practce CS09 Dec 5, 08 Practce Fal Examato Solutos. Aswer: 4/5 8/7. There are multle ways to obta ths aswer; here are two: The frst commo method s to sum over all ossbltes for the rak of
More informationLecture 4 Sep 9, 2015
CS 388R: Radomzed Algorthms Fall 205 Prof. Erc Prce Lecture 4 Sep 9, 205 Scrbe: Xagru Huag & Chad Voegele Overvew I prevous lectures, we troduced some basc probablty, the Cheroff boud, the coupo collector
More informationCS286.2 Lecture 4: Dinur s Proof of the PCP Theorem
CS86. Lecture 4: Dur s Proof of the PCP Theorem Scrbe: Thom Bohdaowcz Prevously, we have prove a weak verso of the PCP theorem: NP PCP 1,1/ (r = poly, q = O(1)). Wth ths result we have the desred costat
More informationMu Sequences/Series Solutions National Convention 2014
Mu Sequeces/Seres Solutos Natoal Coveto 04 C 6 E A 6C A 6 B B 7 A D 7 D C 7 A B 8 A B 8 A C 8 E 4 B 9 B 4 E 9 B 4 C 9 E C 0 A A 0 D B 0 C C Usg basc propertes of arthmetc sequeces, we fd a ad bm m We eed
More informationThe Occupancy and Coupon Collector problems
Chapter 4 The Occupacy ad Coupo Collector problems By Sarel Har-Peled, Jauary 9, 08 4 Prelmares [ Defto 4 Varace ad Stadard Devato For a radom varable X, let V E [ X [ µ X deote the varace of X, where
More informationLecture 9. Some Useful Discrete Distributions. Some Useful Discrete Distributions. The observations generated by different experiments have
NM 7 Lecture 9 Some Useful Dscrete Dstrbutos Some Useful Dscrete Dstrbutos The observatos geerated by dfferet eermets have the same geeral tye of behavor. Cosequetly, radom varables assocated wth these
More informationChapter 5 Properties of a Random Sample
Lecture 3 o BST 63: Statstcal Theory I Ku Zhag, /6/006 Revew for the revous lecture Cocets: radom samle, samle mea, samle varace Theorems: roertes of a radom samle, samle mea, samle varace Examles: how
More informationIntroduction to Probability
Itroducto to Probablty Nader H Bshouty Departmet of Computer Scece Techo 32000 Israel e-mal: bshouty@cstechoacl 1 Combatorcs 11 Smple Rules I Combatorcs The rule of sum says that the umber of ways to choose
More informationBounds on the expected entropy and KL-divergence of sampled multinomial distributions. Brandon C. Roy
Bouds o the expected etropy ad KL-dvergece of sampled multomal dstrbutos Brado C. Roy bcroy@meda.mt.edu Orgal: May 18, 2011 Revsed: Jue 6, 2011 Abstract Iformato theoretc quattes calculated from a sampled
More informationρ < 1 be five real numbers. The
Lecture o BST 63: Statstcal Theory I Ku Zhag, /0/006 Revew for the prevous lecture Deftos: covarace, correlato Examples: How to calculate covarace ad correlato Theorems: propertes of correlato ad covarace
More informationSTK3100 and STK4100 Autumn 2017
SK3 ad SK4 Autum 7 Geeralzed lear models Part III Covers the followg materal from chaters 4 ad 5: Sectos 4..5, 4.3.5, 4.3.6, 4.4., 4.4., ad 4.4.3 Sectos 5.., 5.., ad 5.5. Ørulf Borga Deartmet of Mathematcs
More information{ }{ ( )} (, ) = ( ) ( ) ( ) Chapter 14 Exercises in Sampling Theory. Exercise 1 (Simple random sampling): Solution:
Chapter 4 Exercses Samplg Theory Exercse (Smple radom samplg: Let there be two correlated radom varables X ad A sample of sze s draw from a populato by smple radom samplg wthout replacemet The observed
More information9 U-STATISTICS. Eh =(m!) 1 Eh(X (1),..., X (m ) ) i.i.d
9 U-STATISTICS Suppose,,..., are P P..d. wth CDF F. Our goal s to estmate the expectato t (P)=Eh(,,..., m ). Note that ths expectato requres more tha oe cotrast to E, E, or Eh( ). Oe example s E or P((,
More informationDiscrete Mathematics and Probability Theory Fall 2016 Seshia and Walrand DIS 10b
CS 70 Dscrete Mathematcs ad Probablty Theory Fall 206 Sesha ad Walrad DIS 0b. Wll I Get My Package? Seaky delvery guy of some compay s out delverg packages to customers. Not oly does he had a radom package
More information= lim. (x 1 x 2... x n ) 1 n. = log. x i. = M, n
.. Soluto of Problem. M s obvously cotuous o ], [ ad ], [. Observe that M x,..., x ) M x,..., x ) )..) We ext show that M s odecreasg o ], [. Of course.) mles that M s odecreasg o ], [ as well. To show
More informationCHAPTER 6. d. With success = observation greater than 10, x = # of successes = 4, and
CHAPTR 6 Secto 6.. a. We use the samle mea, to estmate the oulato mea µ. Σ 9.80 µ 8.407 7 ~ 7. b. We use the samle meda, 7 (the mddle observato whe arraged ascedg order. c. We use the samle stadard devato,
More information2SLS Estimates ECON In this case, begin with the assumption that E[ i
SLS Estmates ECON 3033 Bll Evas Fall 05 Two-Stage Least Squares (SLS Cosder a stadard lear bvarate regresso model y 0 x. I ths case, beg wth the assumto that E[ x] 0 whch meas that OLS estmates of wll
More informationMATH 247/Winter Notes on the adjoint and on normal operators.
MATH 47/Wter 00 Notes o the adjot ad o ormal operators I these otes, V s a fte dmesoal er product space over, wth gve er * product uv, T, S, T, are lear operators o V U, W are subspaces of V Whe we say
More information5 Short Proofs of Simplified Stirling s Approximation
5 Short Proofs of Smplfed Strlg s Approxmato Ofr Gorodetsky, drtymaths.wordpress.com Jue, 20 0 Itroducto Strlg s approxmato s the followg (somewhat surprsg) approxmato of the factoral,, usg elemetary fuctos:
More informationUNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Postpoed exam: ECON430 Statstcs Date of exam: Jauary 0, 0 Tme for exam: 09:00 a.m. :00 oo The problem set covers 5 pages Resources allowed: All wrtte ad prted
More informationmeans the first term, a2 means the term, etc. Infinite Sequences: follow the same pattern forever.
9.4 Sequeces ad Seres Pre Calculus 9.4 SEQUENCES AND SERIES Learg Targets:. Wrte the terms of a explctly defed sequece.. Wrte the terms of a recursvely defed sequece. 3. Determe whether a sequece s arthmetc,
More informationPTAS for Bin-Packing
CS 663: Patter Matchg Algorthms Scrbe: Che Jag /9/00. Itroducto PTAS for B-Packg The B-Packg problem s NP-hard. If we use approxmato algorthms, the B-Packg problem could be solved polyomal tme. For example,
More informationEntropy, Relative Entropy and Mutual Information
Etro Relatve Etro ad Mutual Iformato rof. Ja-Lg Wu Deartmet of Comuter Scece ad Iformato Egeerg Natoal Tawa Uverst Defto: The Etro of a dscrete radom varable s defed b : base : 0 0 0 as bts 0 : addg terms
More informationSTK4011 and STK9011 Autumn 2016
STK4 ad STK9 Autum 6 Pot estmato Covers (most of the followg materal from chapter 7: Secto 7.: pages 3-3 Secto 7..: pages 3-33 Secto 7..: pages 35-3 Secto 7..3: pages 34-35 Secto 7.3.: pages 33-33 Secto
More informationDimensionality Reduction and Learning
CMSC 35900 (Sprg 009) Large Scale Learg Lecture: 3 Dmesoalty Reducto ad Learg Istructors: Sham Kakade ad Greg Shakharovch L Supervsed Methods ad Dmesoalty Reducto The theme of these two lectures s that
More informationCS 2750 Machine Learning Lecture 5. Density estimation. Density estimation
CS 750 Mache Learg Lecture 5 esty estmato Mlos Hausrecht mlos@tt.edu 539 Seott Square esty estmato esty estmato: s a usuervsed learg roblem Goal: Lear a model that rereset the relatos amog attrbutes the
More informationUnit 9. The Tangent Bundle
Ut 9. The Taget Budle ========================================================================================== ---------- The taget sace of a submafold of R, detfcato of taget vectors wth dervatos at
More informationParameter, Statistic and Random Samples
Parameter, Statstc ad Radom Samples A parameter s a umber that descrbes the populato. It s a fxed umber, but practce we do ot kow ts value. A statstc s a fucto of the sample data,.e., t s a quatty whose
More informationEconometric Methods. Review of Estimation
Ecoometrc Methods Revew of Estmato Estmatg the populato mea Radom samplg Pot ad terval estmators Lear estmators Ubased estmators Lear Ubased Estmators (LUEs) Effcecy (mmum varace) ad Best Lear Ubased Estmators
More informationLecture 3 Probability review (cont d)
STATS 00: Itroducto to Statstcal Iferece Autum 06 Lecture 3 Probablty revew (cot d) 3. Jot dstrbutos If radom varables X,..., X k are depedet, the ther dstrbuto may be specfed by specfyg the dvdual dstrbuto
More informationSpecial Instructions / Useful Data
JAM 6 Set of all real umbers P A..d. B, p Posso Specal Istructos / Useful Data x,, :,,, x x Probablty of a evet A Idepedetly ad detcally dstrbuted Bomal dstrbuto wth parameters ad p Posso dstrbuto wth
More informationRandom Variables and Probability Distributions
Radom Varables ad Probablty Dstrbutos * If X : S R s a dscrete radom varable wth rage {x, x, x 3,. } the r = P (X = xr ) = * Let X : S R be a dscrete radom varable wth rage {x, x, x 3,.}.If x r P(X = x
More informationX ε ) = 0, or equivalently, lim
Revew for the prevous lecture Cocepts: order statstcs Theorems: Dstrbutos of order statstcs Examples: How to get the dstrbuto of order statstcs Chapter 5 Propertes of a Radom Sample Secto 55 Covergece
More informationApplication of Generating Functions to the Theory of Success Runs
Aled Mathematcal Sceces, Vol. 10, 2016, o. 50, 2491-2495 HIKARI Ltd, www.m-hkar.com htt://dx.do.org/10.12988/ams.2016.66197 Alcato of Geeratg Fuctos to the Theory of Success Rus B.M. Bekker, O.A. Ivaov
More informationChapter 4 Multiple Random Variables
Revew for the prevous lecture: Theorems ad Examples: How to obta the pmf (pdf) of U = g (, Y) ad V = g (, Y) Chapter 4 Multple Radom Varables Chapter 44 Herarchcal Models ad Mxture Dstrbutos Examples:
More informationOrdinary Least Squares Regression. Simple Regression. Algebra and Assumptions.
Ordary Least Squares egresso. Smple egresso. Algebra ad Assumptos. I ths part of the course we are gog to study a techque for aalysg the lear relatoshp betwee two varables Y ad X. We have pars of observatos
More informationFactorization of Finite Abelian Groups
Iteratoal Joural of Algebra, Vol 6, 0, o 3, 0-07 Factorzato of Fte Abela Grous Khald Am Uversty of Bahra Deartmet of Mathematcs PO Box 3038 Sakhr, Bahra kamee@uobedubh Abstract If G s a fte abela grou
More informationThe Arithmetic-Geometric mean inequality in an external formula. Yuki Seo. October 23, 2012
Sc. Math. Japocae Vol. 00, No. 0 0000, 000 000 1 The Arthmetc-Geometrc mea equalty a exteral formula Yuk Seo October 23, 2012 Abstract. The classcal Jese equalty ad ts reverse are dscussed by meas of terally
More informationThird handout: On the Gini Index
Thrd hadout: O the dex Corrado, a tala statstca, proposed (, 9, 96) to measure absolute equalt va the mea dfferece whch s defed as ( / ) where refers to the total umber of dvduals socet. Assume that. The
More informationComplete Convergence and Some Maximal Inequalities for Weighted Sums of Random Variables
Joural of Sceces, Islamc Republc of Ira 8(4): -6 (007) Uversty of Tehra, ISSN 06-04 http://sceces.ut.ac.r Complete Covergece ad Some Maxmal Iequaltes for Weghted Sums of Radom Varables M. Am,,* H.R. Nl
More informationSTK3100 and STK4100 Autumn 2018
SK3 ad SK4 Autum 8 Geeralzed lear models Part III Covers the followg materal from chaters 4 ad 5: Cofdece tervals by vertg tests Cosder a model wth a sgle arameter β We may obta a ( α% cofdece terval for
More informationClass 13,14 June 17, 19, 2015
Class 3,4 Jue 7, 9, 05 Pla for Class3,4:. Samplg dstrbuto of sample mea. The Cetral Lmt Theorem (CLT). Cofdece terval for ukow mea.. Samplg Dstrbuto for Sample mea. Methods used are based o CLT ( Cetral
More informationIntroduction to local (nonparametric) density estimation. methods
Itroducto to local (oparametrc) desty estmato methods A slecture by Yu Lu for ECE 66 Sprg 014 1. Itroducto Ths slecture troduces two local desty estmato methods whch are Parze desty estmato ad k-earest
More informationLaw of Large Numbers
Toss a co tmes. Law of Large Numbers Suppose 0 f f th th toss came up H toss came up T s are Beroull radom varables wth p ½ ad E( ) ½. The proporto of heads s. Itutvely approaches ½ as. week 2 Markov s
More informationMedian as a Weighted Arithmetic Mean of All Sample Observations
Meda as a Weghted Arthmetc Mea of All Sample Observatos SK Mshra Dept. of Ecoomcs NEHU, Shllog (Ida). Itroducto: Iumerably may textbooks Statstcs explctly meto that oe of the weakesses (or propertes) of
More information1 Onto functions and bijections Applications to Counting
1 Oto fuctos ad bectos Applcatos to Coutg Now we move o to a ew topc. Defto 1.1 (Surecto. A fucto f : A B s sad to be surectve or oto f for each b B there s some a A so that f(a B. What are examples of
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Aalyss of Varace ad Desg of Exermets-I MODULE II LECTURE - GENERAL LINEAR HYPOTHESIS AND ANALYSIS OF VARIANCE Dr Shalabh Deartmet of Mathematcs ad Statstcs Ida Isttute of Techology Kaur Tukey s rocedure
More informationCHAPTER VI Statistical Analysis of Experimental Data
Chapter VI Statstcal Aalyss of Expermetal Data CHAPTER VI Statstcal Aalyss of Expermetal Data Measuremets do ot lead to a uque value. Ths s a result of the multtude of errors (maly radom errors) that ca
More information1 Solution to Problem 6.40
1 Soluto to Problem 6.40 (a We wll wrte T τ (X 1,...,X where the X s are..d. wth PDF f(x µ, σ 1 ( x µ σ g, σ where the locato parameter µ s ay real umber ad the scale parameter σ s > 0. Lettg Z X µ σ we
More informationMA 524 Homework 6 Solutions
MA 524 Homework 6 Solutos. Sce S(, s the umber of ways to partto [] to k oempty blocks, ad c(, s the umber of ways to partto to k oempty blocks ad also the arrage each block to a cycle, we must have S(,
More informationIS 709/809: Computational Methods in IS Research. Simple Markovian Queueing Model
IS 79/89: Comutatoal Methods IS Research Smle Marova Queueg Model Nrmalya Roy Deartmet of Iformato Systems Uversty of Marylad Baltmore Couty www.umbc.edu Queueg Theory Software QtsPlus software The software
More informationProbability and Statistics. What is probability? What is statistics?
robablt ad Statstcs What s robablt? What s statstcs? robablt ad Statstcs robablt Formall defed usg a set of aoms Seeks to determe the lkelhood that a gve evet or observato or measuremet wll or has haeed
More informationChapter 5 Properties of a Random Sample
Lecture 6 o BST 63: Statstcal Theory I Ku Zhag, /0/008 Revew for the prevous lecture Cocepts: t-dstrbuto, F-dstrbuto Theorems: Dstrbutos of sample mea ad sample varace, relatoshp betwee sample mea ad sample
More informationMATH 371 Homework assignment 1 August 29, 2013
MATH 371 Homework assgmet 1 August 29, 2013 1. Prove that f a subset S Z has a smallest elemet the t s uque ( other words, f x s a smallest elemet of S ad y s also a smallest elemet of S the x y). We kow
More information,m = 1,...,n; 2 ; p m (1 p) n m,m = 0,...,n; E[X] = np; n! e λ,n 0; E[X] = λ.
CS70: Lecture 21. Revew: Dstrbutos Revew: Idepedece Varace; Iequaltes; WLLN 1. Revew: Dstrbutos 2. Revew: Idepedece 3. Varace 4. Iequaltes Markov Chebyshev 5. Weak Law of Large Numbers U[1,...,] : Pr[X
More informationOn the introductory notes on Artin s Conjecture
O the troductory otes o Art s Cojecture The urose of ths ote s to make the surveys [5 ad [6 more accessble to bachelor studets. We rovde some further relmares ad some exercses. We also reset the calculatos
More informationChapter 8: Statistical Analysis of Simulated Data
Marquette Uversty MSCS600 Chapter 8: Statstcal Aalyss of Smulated Data Dael B. Rowe, Ph.D. Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 08 by Marquette Uversty MSCS600 Ageda 8. The Sample
More informationThe Mathematical Appendix
The Mathematcal Appedx Defto A: If ( Λ, Ω, where ( λ λ λ whch the probablty dstrbutos,,..., Defto A. uppose that ( Λ,,..., s a expermet type, the σ-algebra o λ λ λ are defed s deoted by ( (,,...,, σ Ω.
More information18.413: Error Correcting Codes Lab March 2, Lecture 8
18.413: Error Correctg Codes Lab March 2, 2004 Lecturer: Dael A. Spelma Lecture 8 8.1 Vector Spaces A set C {0, 1} s a vector space f for x all C ad y C, x + y C, where we take addto to be compoet wse
More informationTwo Fuzzy Probability Measures
Two Fuzzy robablty Measures Zdeěk Karíšek Isttute of Mathematcs Faculty of Mechacal Egeerg Bro Uversty of Techology Techcká 2 66 69 Bro Czech Reublc e-mal: karsek@umfmevutbrcz Karel Slavíček System dmstrato
More informationUNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Exam: ECON430 Statstcs Date of exam: Frday, December 8, 07 Grades are gve: Jauary 4, 08 Tme for exam: 0900 am 00 oo The problem set covers 5 pages Resources allowed:
More informationPROJECTION PROBLEM FOR REGULAR POLYGONS
Joural of Mathematcal Sceces: Advaces ad Applcatos Volume, Number, 008, Pages 95-50 PROJECTION PROBLEM FOR REGULAR POLYGONS College of Scece Bejg Forestry Uversty Bejg 0008 P. R. Cha e-mal: sl@bjfu.edu.c
More informationBounds for the Connective Eccentric Index
It. J. Cotemp. Math. Sceces, Vol. 7, 0, o. 44, 6-66 Bouds for the Coectve Eccetrc Idex Nlaja De Departmet of Basc Scece, Humates ad Socal Scece (Mathematcs Calcutta Isttute of Egeerg ad Maagemet Kolkata,
More informationCIS 800/002 The Algorithmic Foundations of Data Privacy October 13, Lecture 9. Database Update Algorithms: Multiplicative Weights
CIS 800/002 The Algorthmc Foudatos of Data Prvacy October 13, 2011 Lecturer: Aaro Roth Lecture 9 Scrbe: Aaro Roth Database Update Algorthms: Multplcatve Weghts We ll recall aga) some deftos from last tme:
More informationNon-uniform Turán-type problems
Joural of Combatoral Theory, Seres A 111 2005 106 110 wwwelsevercomlocatecta No-uform Turá-type problems DhruvMubay 1, Y Zhao 2 Departmet of Mathematcs, Statstcs, ad Computer Scece, Uversty of Illos at
More information18.657: Mathematics of Machine Learning
8.657: Mathematcs of Mache Learg Lecturer: Phlppe Rgollet Lecture 3 Scrbe: James Hrst Sep. 6, 205.5 Learg wth a fte dctoary Recall from the ed of last lecture our setup: We are workg wth a fte dctoary
More informationChannel Models with Memory. Channel Models with Memory. Channel Models with Memory. Channel Models with Memory
Chael Models wth Memory Chael Models wth Memory Hayder radha Electrcal ad Comuter Egeerg Mchga State Uversty I may ractcal etworkg scearos (cludg the Iteret ad wreless etworks), the uderlyg chaels are
More informationSTRONG CONSISTENCY FOR SIMPLE LINEAR EV MODEL WITH v/ -MIXING
Joural of tatstcs: Advaces Theory ad Alcatos Volume 5, Number, 6, Pages 3- Avalable at htt://scetfcadvaces.co. DOI: htt://d.do.org/.864/jsata_7678 TRONG CONITENCY FOR IMPLE LINEAR EV MODEL WITH v/ -MIXING
More informationTHE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA
THE ROYAL STATISTICAL SOCIETY 3 EXAMINATIONS SOLUTIONS GRADUATE DIPLOMA PAPER I STATISTICAL THEORY & METHODS The Socety provdes these solutos to assst caddates preparg for the examatos future years ad
More informationHomework 1: Solutions Sid Banerjee Problem 1: (Practice with Asymptotic Notation) ORIE 4520: Stochastics at Scale Fall 2015
Fall 05 Homework : Solutos Problem : (Practce wth Asymptotc Notato) A essetal requremet for uderstadg scalg behavor s comfort wth asymptotc (or bg-o ) otato. I ths problem, you wll prove some basc facts
More informationarxiv:math/ v1 [math.gm] 8 Dec 2005
arxv:math/05272v [math.gm] 8 Dec 2005 A GENERALIZATION OF AN INEQUALITY FROM IMO 2005 NIKOLAI NIKOLOV The preset paper was spred by the thrd problem from the IMO 2005. A specal award was gve to Yure Boreko
More informationd dt d d dt dt Also recall that by Taylor series, / 2 (enables use of sin instead of cos-see p.27 of A&F) dsin
Learzato of the Swg Equato We wll cover sectos.5.-.6 ad begg of Secto 3.3 these otes. 1. Sgle mache-fte bus case Cosder a sgle mache coected to a fte bus, as show Fg. 1 below. E y1 V=1./_ Fg. 1 The admttace
More informationMinkowski s inequality and sums of squares
Cet Eur J Math 13 014 510-516 DOI: 10478/s11533-013-0346-1 Cetral Euroea Joural of Mathematcs Mows s equalty ad sums of squares Research Artcle Péter E Freel 1, Péter Horváth 1 1 Deartmet of Algebra ad
More informationLecture 9: Tolerant Testing
Lecture 9: Tolerat Testg Dael Kae Scrbe: Sakeerth Rao Aprl 4, 07 Abstract I ths lecture we prove a quas lear lower boud o the umber of samples eeded to do tolerat testg for L dstace. Tolerat Testg We have
More informationFunctions of Random Variables
Fuctos of Radom Varables Chapter Fve Fuctos of Radom Varables 5. Itroducto A geeral egeerg aalyss model s show Fg. 5.. The model output (respose) cotas the performaces of a system or product, such as weght,
More informationNonparametric Density Estimation Intro
Noarametrc Desty Estmato Itro Parze Wdows No-Parametrc Methods Nether robablty dstrbuto or dscrmat fucto s kow Haes qute ofte All we have s labeled data a lot s kow easer salmo bass salmo salmo Estmate
More informationSemi-Riemann Metric on. the Tangent Bundle and its Index
t J Cotem Math Sceces ol 5 o 3 33-44 Sem-Rema Metrc o the Taet Budle ad ts dex smet Ayha Pamuale Uversty Educato Faculty Dezl Turey ayha@auedutr Erol asar Mers Uversty Art ad Scece Faculty 33343 Mers Turey
More informationX X X E[ ] E X E X. is the ()m n where the ( i,)th. j element is the mean of the ( i,)th., then
Secto 5 Vectors of Radom Varables Whe workg wth several radom varables,,..., to arrage them vector form x, t s ofte coveet We ca the make use of matrx algebra to help us orgaze ad mapulate large umbers
More informationLecture 3 Naïve Bayes, Maximum Entropy and Text Classification COSI 134
Lecture 3 Naïve Baes, Mamum Etro ad Tet Classfcato COSI 34 Codtoal Parameterzato Two RVs: ItellgeceI ad SATS ValI = {Hgh,Low}, ValS={Hgh,Low} A ossble jot dstrbuto Ca descrbe usg cha rule as PI,S PIPS
More informationChapter 4 Multiple Random Variables
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 Revew for Chapter 4-5 Notes: Although all deftos ad theorems troduced our lectures ad ths ote are mportat ad you should be famlar wth, but I put those
More informationResearch Article A New Iterative Method for Common Fixed Points of a Finite Family of Nonexpansive Mappings
Hdaw Publshg Corporato Iteratoal Joural of Mathematcs ad Mathematcal Sceces Volume 009, Artcle ID 391839, 9 pages do:10.1155/009/391839 Research Artcle A New Iteratve Method for Commo Fxed Pots of a Fte
More information#A27 INTEGERS 13 (2013) SOME WEIGHTED SUMS OF PRODUCTS OF LUCAS SEQUENCES
#A27 INTEGERS 3 (203) SOME WEIGHTED SUMS OF PRODUCTS OF LUCAS SEQUENCES Emrah Kılıç Mathematcs Departmet, TOBB Uversty of Ecoomcs ad Techology, Akara, Turkey eklc@etu.edu.tr Neşe Ömür Mathematcs Departmet,
More informationMultiple Choice Test. Chapter Adequacy of Models for Regression
Multple Choce Test Chapter 06.0 Adequac of Models for Regresso. For a lear regresso model to be cosdered adequate, the percetage of scaled resduals that eed to be the rage [-,] s greater tha or equal to
More informationBASIC PRINCIPLES OF STATISTICS
BASIC PRINCIPLES OF STATISTICS PROBABILITY DENSITY DISTRIBUTIONS DISCRETE VARIABLES BINOMIAL DISTRIBUTION ~ B 0 0 umber of successes trals Pr E [ ] Var[ ] ; BINOMIAL DISTRIBUTION B7 0. B30 0.3 B50 0.5
More informationLecture Notes to Rice Chapter 5
ECON 430 Revsed Sept. 06 Lecture Notes to Rce Chapter 5 By H. Goldste. Chapter 5 gves a troducto to probablstc approxmato methods, but s suffcet for the eeds of a adequate study of ecoometrcs. The commo
More informationTHE PUBLISHING HOUSE PROCEEDINGS OF THE ROMANIAN ACADEMY, Series A, OF THE ROMANIAN ACADEMY Volume 9, Number 3/2008, pp
THE PUBLISHIN HOUSE PROCEEDINS OF THE ROMANIAN ACADEMY, Seres A, OF THE ROMANIAN ACADEMY Volume 9, Number 3/8, THE UNITS IN Stela Corelu ANDRONESCU Uversty of Pteşt, Deartmet of Mathematcs, Târgu Vale
More informationAN UPPER BOUND FOR THE PERMANENT VERSUS DETERMINANT PROBLEM BRUNO GRENET
AN UPPER BOUND FOR THE PERMANENT VERSUS DETERMINANT PROBLEM BRUNO GRENET Abstract. The Permaet versus Determat problem s the followg: Gve a matrx X of determates over a feld of characterstc dfferet from
More informationLecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model
Lecture 7. Cofdece Itervals ad Hypothess Tests the Smple CLR Model I lecture 6 we troduced the Classcal Lear Regresso (CLR) model that s the radom expermet of whch the data Y,,, K, are the outcomes. The
More informationTESTS BASED ON MAXIMUM LIKELIHOOD
ESE 5 Toy E. Smth. The Basc Example. TESTS BASED ON MAXIMUM LIKELIHOOD To llustrate the propertes of maxmum lkelhood estmates ad tests, we cosder the smplest possble case of estmatg the mea of the ormal
More informationON BIVARIATE GEOMETRIC DISTRIBUTION. K. Jayakumar, D.A. Mundassery 1. INTRODUCTION
STATISTICA, ao LXVII, 4, 007 O BIVARIATE GEOMETRIC DISTRIBUTIO ITRODUCTIO Probablty dstrbutos of radom sums of deedetly ad detcally dstrbuted radom varables are maly aled modelg ractcal roblems that deal
More informationTHE PROBABILISTIC STABILITY FOR THE GAMMA FUNCTIONAL EQUATION
Joural of Scece ad Arts Year 12, No. 3(2), pp. 297-32, 212 ORIGINAL AER THE ROBABILISTIC STABILITY FOR THE GAMMA FUNCTIONAL EQUATION DOREL MIHET 1, CLAUDIA ZAHARIA 1 Mauscrpt receved: 3.6.212; Accepted
More informationGeneralization of the Dissimilarity Measure of Fuzzy Sets
Iteratoal Mathematcal Forum 2 2007 o. 68 3395-3400 Geeralzato of the Dssmlarty Measure of Fuzzy Sets Faramarz Faghh Boformatcs Laboratory Naobotechology Research Ceter vesa Research Isttute CECR Tehra
More informationChapter 3 Sampling For Proportions and Percentages
Chapter 3 Samplg For Proportos ad Percetages I may stuatos, the characterstc uder study o whch the observatos are collected are qualtatve ature For example, the resposes of customers may marketg surveys
More informationAlgorithms Design & Analysis. Hash Tables
Algorthms Desg & Aalyss Hash Tables Recap Lower boud Order statstcs 2 Today s topcs Drect-accessble table Hash tables Hash fuctos Uversal hashg Perfect Hashg Ope addressg 3 Symbol-table problem Symbol
More informationIntegral Generalized Binomial Coefficients of Multiplicative Functions
Uversty of Puget Soud Soud Ideas Summer Research Summer 015 Itegral Geeralzed Bomal Coeffcets of Multlcatve Fuctos Imauel Che hche@ugetsoud.edu Follow ths ad addtoal works at: htt://souddeas.ugetsoud.edu/summer_research
More information