Supplementary Material for Limits on Sparse Support Recovery via Linear Sketching with Random Expander Matrices
|
|
- Rolf Richards
- 6 years ago
- Views:
Transcription
1 Joata Scarlett ad Volka Cever Supplemetary Materal for Lmts o Sparse Support Recovery va Lear Sketcg wt Radom Expader Matrces (AISTATS 26, Joata Scarlett ad Volka Cever) Note tat all ctatos ere are to te bblograpy te ma documet, ad smlarly for may of te crossrefereces. A Proof of Lemma I te otato of Defto, let E ( =,...,k)beteevettatsomesets of cardalty fals to satsfy te expaso property,.e., N (S) < ( )d S. We start wt te followg o-asymptotc boud gve 8]: p d d PE] d. (44) d Applyg te bouds log p log p ad log d d dh 2 ( ), weobta log PE] log p + dh 2 ( )+ d log d (45) = log p d log H 2 ( ). (46) d Sce k = (), weobtafromteuoboudtatp =,...,k E s! provded tat (46) teds to all. Ts s true provded tat (2) olds; te domat codto s te oe wt = k. for B Proof of Teorem 3 Recall te deftos of te radom varables () (), ad te formato destes (25) (27). We fx te costats,..., k arbtrarly, ad cosder a decoder tat searces for te uque set s 2S suc tat for all 2 k declared. ĩ(x sdf ; y x seq ) > sdf (47) parttos (s df,s eq ) of s wt s df 6= ;. If o suc s exsts, or f multple exst, te a error s Sce te jot dstrbuto of ( s, s, Y s S = s) s te same for all s our setup (cf., Secto.2), ad te decoder tat we ave cose exbts a smlar symmetry, we ca codto o S = s = {,...,k}. Byteuo boud, te error probablty s upper bouded by P e P o ĩ( sdf ; Y seq ) sdf + s2s\{s} P ĩ( s\s ; Y s\s ) > sdf, (48) were ere ad subsequetly we let te codto s df 6= ; rema mplct. I te summad of te secod term, we ave upper bouded te probablty of a tersecto of 2 k evets by just oe suc evet, amely, te oe wt te formato desty correspodg to s df = s\s ad s eq = s \ s. As metoed prevously, a key tool te proof s te followg cage of measure (wt := s df ): P Y seq (y x seq )= Y P (x ) P Y sdf seq (y x sdf, x seq ) (49) x sdf 2s df ( + Y ) P(x ) P Y sdf seq (y x sdf, x seq ) (5) 2s df x sdf =( + ) e PY seq (y x seq ), (5) were we ave used te deftos (23) (24), ad (5) follows from (2). By a detcal argumet, we ave P Y seq s (y x seq,b s ) ( + ) e PY seq s (y x seq,b s ), (52)
2 Lmts o Sparse Support Recovery va Lear Sketcg wt Radom Expader Matrces were e P Y seq s := P Y seq s as a..d. law. We ca weake te secod probablty (48) as follows (wt := s\s ): P ĩ( s\s ; Y s\s ) > = P k (x s\s )P (x s\s ) dy P Y seq (y x s\s ) log P Y sdf seq (y x s\s, x s\s ) > (53) x s\s,x R s\s ep Y seq (y x s\s ) ( + ) (x s\s )P (x s\s ) dy P e Y seq (y x s\s ) log P Y sdf seq (y x s\s, x s\s ) > R ep Y seq (y x s\s ) ( + ) x s\s,x s\s P k =( + )e, x s\s,x s\s P k (54) (x s\s )P (x s\s ) dy P Y sdf seq (y x s\s, x s\s )e (55) R were (53) we used te fact tat te output vector depeds oly o te colums of x s correspodg to etres of s tat are also s, (54) follows from (5), ad (55) follows by boudg P e Y seq usg te evet wt te dcator fucto, ad te upper boudg te dcator fucto by oe. Substtutg (56) to (48) gves P e P ĩ( sdf ; Y seq ) were te combatoral terms arse from a stadard coutg argumet 7]. o + = (56) k p k k ( + )e, (57) We ow fx te costats,..., k arbtrarly, ad recall te followg steps from 7] (aga wrtg := s df ): o P ĩ( sdf ; Y seq ) = P P P log P Y sdf seq (Y sdf, seq ) (58) ep Y seq (Y seq ) log P Y sdf seq (Y sdf, seq ) ep Y seq (Y seq ) Te secod term (6) s upper bouded as P = log P log + P \ log log P Y sdf seq (Y sdf, seq ) + (Y seq, s) ep Y seq (Y seq ) (Y seq, s) > + P log log ep Y seq (Y seq ) (Y seq, s) ep Y seq (Y seq ) (Y seq, s) > (59) ep Y seq (Y seq ) (Y seq, s) >. (6) ep Y seq (Y seq ) (Y seq, s) > (6) P s (b s )P k (x seq ) dy P Y seq s (y x seq,b s ) log b R s,x seq ep Y seq (y x seq ) (y x seq,b s ) > (62)
3 ( + ) ( + ) =( + ) k = k e Joata Scarlett ad Volka Cever P s (b s )P k (x seq ) dy e ep Y seq (y x seq ) P Y seq s (y x seq,b s ) log b R s,x seq (y x seq,b s ) > P s (b s )P k (x seq ) dy P e Y seq (y x seq )e b R s,x seq, (63) (64) were (6) follows from te uo boud, ad te remag steps follow te argumets used (53) (56) (wt (52) used place of (5)). We ow upper boud te frst term (6), aga followg 7]. Te umerator te frst term (6) equals P Y s (Y s ) for all (s df,s eq ) (recall te defto (22)), ad we ca tus wrte te overall term as P log P Y s (Y s ) max Usg te same steps as tose used (58) (6), we ca upper boud ts by P log P Y s s (Y s, s) max for ay costat P (65) log e P Y seq s (Y seq, s)+ +. (66) log e P Y seq s (Y seq, s) Reversg te step (66), ts ca equvaletly be wrtte as + P log P Y s, s (Y s, s) > (67) P Y s (Y s ) log P Y sdf seq s (Y sdf, seq, s) P log P Y s, s (Y s, s) >. (68) (Y seq, s) P Y s (Y s ) Te frst logartm te frst term s te formato desty (26). Moreover, te coces k p k k = log ( + ) k k = log ( + ) (69) (7) make (65) ad te secod term (57) be upper bouded by (68), ad recallg tat = s df,weobta(28). eac. Hece, ad combg (6) wt (65) ad C Proof of Teorem 2 Fx <b m <b max <, adletb := {b s :m b b m \ max b b max }. Te ma step provg Teorem 2 s extedg te argumets of Secto 4.5 to sow tat s df log p P e P max :s df 6=; I sdf,s eq ( s ) ( + ) \ s 2B + P s /2 B + o(), (7) ad P e P max :s df 6=; s df log p I sdf,s eq ( s ) ( ) \ s 2B + o(), (72) Before provg tese, we sow ow tey yeld te teorem. Usg (6), t s readly verfed tat eac I sdf,s eq ( s ), wt a..d. Gaussa vector s, sacotuousradomvarableavgomasspots. Bytakg! suffcetly slowly ad otg tat we ave restrcted s to te set B (wt wc all of te I sdf,s eq ( s ) are
4 Lmts o Sparse Support Recovery va Lear Sketcg wt Radom Expader Matrces bouded away from zero ad fty), we coclude tat (7) (72) rema true we s replaced by zero, ad ts cotrbuto s factored to te o() terms. Hece, we obta Teorem 2 by () droppg te codto s 2B from te frst probablty (7); () usg te detty PA \A 2 ] PA ] PA 2 ] to remove te same codto from te frst probablty (72); () otg tat te remader term P s /2 B ca be made arbtrarly small by coosg b m suffcetly small ad b max suffcetly large. It remas to establs (7) (72). Recall te value of gve followg Lemma 3. Te above coce of B esures tat all of te o-zero etres are bouded away from ad, sotattemutualformatosi sdf,s eq ( s ) ad varaces V sdf,s eq ( s ) are bouded away from zero ad fty, ad ece = (). Sce P s s cotuous, we must coose ad adle P (29) dfferetly to te above. Smlarly to te aalyss of Gaussa measuremets 7], we fx > ad ote tat Cebysev s equalty mples = I + r V =) P ( ), (73) were I := I( s ; Y s ) (74) V := Var log P Y s, s (Y s, s). (75) P Y s (Y s ) Te followg s a stragtforward exteso of 7, Prop. 4] to expader-based measuremets. Proposto. Te quattes I ad V defed (74) (75) satsfy Proof. See Appedx E. I k 2 log + d 2 2 (76) V 2. (77) We ca ow obta (7) (72) usg te steps of te prevous subsecto; te codto P s 2B ] arses (35) ad (39) due to te fact tat ts codto was used to obta a bouded varace (32), ad te frst two probabltes (7) arse from te detty PA A 2 ] PA A c 2]+PA 2 ]. Te oly addtoal step s sowg tat we ca smultaeously aceve = o(log p) ad P ( )=o() te acevablty part weever = (log p), te same way tat we sowed 2 s df log = o(log p) te prevous subsecto. Ts mmedately follows by substtutg (76) (77) to (73) (alog wt d = O() =O(log p)) toobta = O(log log p)+ p log p = o(log p) for ay >, adotgtat (ad ece P ( )) (73)cabearbtrarlysmall. D Proof of Lemma 3 We prove te lemma by caracterzg te varace of a geeral fucto of ( s, Y) of te form f ( s, Y) := P = f(() s,y () ). Clearly all of te quattes ı for te varous (s df,s eq ) ca be wrtte ts geeral form. We ave Var f ( s, Y) = Var f( s (),Y () ) (78) = = = j= Cov f( () s = Var f( s,y) +( 2,Y () ),f( s (j),y (j) ) (79) )Cov f( s,y),f(s,y ), (8) were ( s,y) ad ( s,y ) correspod to two dfferet dces {,,}; ere (8) follows by smple symmetry cosderatos for te cases = j ad 6= j.
5 Joata Scarlett ad Volka Cever To compute te covarace term (8), we frst fd te jot dstrbuto of ( s,y) ad (s,y ). As oted 29, Sec. IV-B], a uform permutato of a vector wt d oes ad d zeros ca be terpreted as successvely performg uform samplg from a collecto of symbols wtout replacemet ( tmes total), were te tal collecto cotas d oes ad d zeros. By cosderg te frst two steps of ts procedure, we obta P = x ]=P (x ) (8) P = x = x ]= P (x ) {x = x } for =, 2, were P () = P () = d. Deotg te rgt-ad sde of (82) by P (x x ), ad wrtg µ f := Ef( s,y)], tecovarace(8)sgveby Cov f( s,y),f(s,y ) = E f( s,y) µ f f(s,y ) µ f (83) = P(x k s ) Y P(x x ) E f(x s,y) µ f f(x s,y ) µ f s = x s,s = x s. (84) x s x s 2s We ow cosder te varous terms arsg by substtutg (82) to (84) ad performg a bomal-type expaso of te product: Tere s a sgle term of te form (84) wt eac P x(x x ) replaced by P (x ). Ts yelds a average of f( s,y) µ f f( s,y ) µ f over depedet radom varables s ad s,adtereforeevaluatesto zero. Tere are k terms wc oe value Px(x x {x ) (84) s replaced by =x } ad te oter k are replaced by P (x ). Eac suc term ca be wrtte as ( ) Var Ef( 2 s,y) s\{} ], wc tur beaves as Var Ef( s,y) s\{} ] + O(). All of te remag terms replace P x(x x ) (84) by {x=x } for at least two values of. Allsucterms are easly verfed to beave as O 2, ad te umber of suc terms s fte ad does ot scale wt (recall tat k s fxed by assumpto). Substtutg tese cases to (84) ad recallg tat k = () ad d = (), weobta(4). E Proof of Proposto Here we caracterze I ad V,defed(74) (75),vaaextesoofteaalyssgve7,App.B]. Sce Y = s s +, weave (82) I = I( s ; Y s )=H(Y s ) H(Y s, s) (85) = H( s s + s ) H(). (86) From 25, C. 9], we ave H() = 2 log(2 e 2 ) ad H( s s + s = x s )= 2 log (2 e) det( 2 I + 2 x s x T s ), were I s te detty matrx. Averagg te latter over s ad substtutg tese to (86) gves I = 2 E log det I + = 2 E log det I k + = 2 k = E log + k 2 log + d s T s 2 2 T s s 2 2 ( T s s ) (87) (88) (89), (9)
6 Lmts o Sparse Support Recovery va Lear Sketcg wt Radom Expader Matrces were (88) follows from te detty det(i + AB) =det(i + BA), (89) follows by wrtg te determat as a product of egevalues (deoted by ( )), ad (9) follows from Jese s equalty ad te followg calculato: k k E = ( T s s ) = k ETr(T s s )] = E T ]=d, (9) sce te squared orm of s d almost surely. Ts cocludes te proof of (76). We ow tur to te boudg of te varace. Aga usg te fact tat Y = s log P Y s, s (Y s, s) P Y s (Y s ) = log P () P Y s ( s s + s ) = I 2 2 T + 2 ( s s + ) T 2 I + 2 s T s s +, weave (92) (s s + ), (93) were P s te desty of, ad (93) follows by a drect substtuto of te destes P N(, 2 I) ad P Y S ( x s ) N(, 2 I + 2 x s x T s ). Observe ow tat 2 T s a sum of depedet 2 radom varables wt oe degree of freedom (eac avg a varace of 2), ad ece te secod term (93) as a varace of 2. Moreover, by wrtg M =(M 2 ) T M 2 for te symmetrc postve defte matrx M = 2 I + 2 s T s,we smlarly observe tat te fal term (93) s a sum of 2 varables (ts s true codtoed o ay s = x s, ad ece also true ucodtoally), aga yeldg a varace of 2. We tus obta (77) usg te detty VarA + B] VarA] + VarB] + 2 max{vara], VarB]}.
TESTS BASED ON MAXIMUM LIKELIHOOD
ESE 5 Toy E. Smth. The Basc Example. TESTS BASED ON MAXIMUM LIKELIHOOD To llustrate the propertes of maxmum lkelhood estmates ad tests, we cosder the smplest possble case of estmatg the mea of the ormal
More informationChapter 4 Multiple Random Variables
Revew for the prevous lecture: Theorems ad Examples: How to obta the pmf (pdf) of U = g (, Y) ad V = g (, Y) Chapter 4 Multple Radom Varables Chapter 44 Herarchcal Models ad Mxture Dstrbutos Examples:
More informationρ < 1 be five real numbers. The
Lecture o BST 63: Statstcal Theory I Ku Zhag, /0/006 Revew for the prevous lecture Deftos: covarace, correlato Examples: How to calculate covarace ad correlato Theorems: propertes of correlato ad covarace
More informationNumerical Analysis Formulae Booklet
Numercal Aalyss Formulae Booklet. Iteratve Scemes for Systems of Lear Algebrac Equatos:.... Taylor Seres... 3. Fte Dfferece Approxmatos... 3 4. Egevalues ad Egevectors of Matrces.... 3 5. Vector ad Matrx
More informationChapter 5 Properties of a Random Sample
Lecture 6 o BST 63: Statstcal Theory I Ku Zhag, /0/008 Revew for the prevous lecture Cocepts: t-dstrbuto, F-dstrbuto Theorems: Dstrbutos of sample mea ad sample varace, relatoshp betwee sample mea ad sample
More informationMu Sequences/Series Solutions National Convention 2014
Mu Sequeces/Seres Solutos Natoal Coveto 04 C 6 E A 6C A 6 B B 7 A D 7 D C 7 A B 8 A B 8 A C 8 E 4 B 9 B 4 E 9 B 4 C 9 E C 0 A A 0 D B 0 C C Usg basc propertes of arthmetc sequeces, we fd a ad bm m We eed
More informationCHAPTER VI Statistical Analysis of Experimental Data
Chapter VI Statstcal Aalyss of Expermetal Data CHAPTER VI Statstcal Aalyss of Expermetal Data Measuremets do ot lead to a uque value. Ths s a result of the multtude of errors (maly radom errors) that ca
More informationENGI 4421 Joint Probability Distributions Page Joint Probability Distributions [Navidi sections 2.5 and 2.6; Devore sections
ENGI 441 Jot Probablty Dstrbutos Page 7-01 Jot Probablty Dstrbutos [Navd sectos.5 ad.6; Devore sectos 5.1-5.] The jot probablty mass fucto of two dscrete radom quattes, s, P ad p x y x y The margal probablty
More information1 Review and Overview
CS9T/STATS3: Statstcal Learg Teory Lecturer: Tegyu Ma Lecture #7 Scrbe: Bra Zag October 5, 08 Revew ad Overvew We wll frst gve a bref revew of wat as bee covered so far I te frst few lectures, we stated
More informationX X X E[ ] E X E X. is the ()m n where the ( i,)th. j element is the mean of the ( i,)th., then
Secto 5 Vectors of Radom Varables Whe workg wth several radom varables,,..., to arrage them vector form x, t s ofte coveet We ca the make use of matrx algebra to help us orgaze ad mapulate large umbers
More informationUNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Postpoed exam: ECON430 Statstcs Date of exam: Jauary 0, 0 Tme for exam: 09:00 a.m. :00 oo The problem set covers 5 pages Resources allowed: All wrtte ad prted
More informationAsymptotic Formulas Composite Numbers II
Iteratoal Matematcal Forum, Vol. 8, 203, o. 34, 65-662 HIKARI Ltd, www.m-kar.com ttp://d.do.org/0.2988/mf.203.3854 Asymptotc Formulas Composte Numbers II Rafael Jakmczuk Dvsó Matemátca, Uversdad Nacoal
More informationMATH 247/Winter Notes on the adjoint and on normal operators.
MATH 47/Wter 00 Notes o the adjot ad o ormal operators I these otes, V s a fte dmesoal er product space over, wth gve er * product uv, T, S, T, are lear operators o V U, W are subspaces of V Whe we say
More informationTHE ROYAL STATISTICAL SOCIETY 2016 EXAMINATIONS SOLUTIONS HIGHER CERTIFICATE MODULE 5
THE ROYAL STATISTICAL SOCIETY 06 EAMINATIONS SOLUTIONS HIGHER CERTIFICATE MODULE 5 The Socety s provdg these solutos to assst cadtes preparg for the examatos 07. The solutos are teded as learg ads ad should
More informationInvestigation of Partially Conditional RP Model with Response Error. Ed Stanek
Partally Codtoal Radom Permutato Model 7- vestgato of Partally Codtoal RP Model wth Respose Error TRODUCTO Ed Staek We explore the predctor that wll result a smple radom sample wth respose error whe a
More informationBounds on the expected entropy and KL-divergence of sampled multinomial distributions. Brandon C. Roy
Bouds o the expected etropy ad KL-dvergece of sampled multomal dstrbutos Brado C. Roy bcroy@meda.mt.edu Orgal: May 18, 2011 Revsed: Jue 6, 2011 Abstract Iformato theoretc quattes calculated from a sampled
More informationX ε ) = 0, or equivalently, lim
Revew for the prevous lecture Cocepts: order statstcs Theorems: Dstrbutos of order statstcs Examples: How to get the dstrbuto of order statstcs Chapter 5 Propertes of a Radom Sample Secto 55 Covergece
More information{ }{ ( )} (, ) = ( ) ( ) ( ) Chapter 14 Exercises in Sampling Theory. Exercise 1 (Simple random sampling): Solution:
Chapter 4 Exercses Samplg Theory Exercse (Smple radom samplg: Let there be two correlated radom varables X ad A sample of sze s draw from a populato by smple radom samplg wthout replacemet The observed
More informationLecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model
Lecture 7. Cofdece Itervals ad Hypothess Tests the Smple CLR Model I lecture 6 we troduced the Classcal Lear Regresso (CLR) model that s the radom expermet of whch the data Y,,, K, are the outcomes. The
More information9 U-STATISTICS. Eh =(m!) 1 Eh(X (1),..., X (m ) ) i.i.d
9 U-STATISTICS Suppose,,..., are P P..d. wth CDF F. Our goal s to estmate the expectato t (P)=Eh(,,..., m ). Note that ths expectato requres more tha oe cotrast to E, E, or Eh( ). Oe example s E or P((,
More informationSummary of the lecture in Biostatistics
Summary of the lecture Bostatstcs Probablty Desty Fucto For a cotuos radom varable, a probablty desty fucto s a fucto such that: 0 dx a b) b a dx A probablty desty fucto provdes a smple descrpto of the
More informationLecture Note to Rice Chapter 8
ECON 430 HG revsed Nov 06 Lecture Note to Rce Chapter 8 Radom matrces Let Y, =,,, m, =,,, be radom varables (r.v. s). The matrx Y Y Y Y Y Y Y Y Y Y = m m m s called a radom matrx ( wth a ot m-dmesoal dstrbuto,
More informationLecture 02: Bounding tail distributions of a random variable
CSCI-B609: A Theorst s Toolkt, Fall 206 Aug 25 Lecture 02: Boudg tal dstrbutos of a radom varable Lecturer: Yua Zhou Scrbe: Yua Xe & Yua Zhou Let us cosder the ubased co flps aga. I.e. let the outcome
More informationChapter 9 Jordan Block Matrices
Chapter 9 Jorda Block atrces I ths chapter we wll solve the followg problem. Gve a lear operator T fd a bass R of F such that the matrx R (T) s as smple as possble. f course smple s a matter of taste.
More informationEconometric Methods. Review of Estimation
Ecoometrc Methods Revew of Estmato Estmatg the populato mea Radom samplg Pot ad terval estmators Lear estmators Ubased estmators Lear Ubased Estmators (LUEs) Effcecy (mmum varace) ad Best Lear Ubased Estmators
More informationChapter 3 Sampling For Proportions and Percentages
Chapter 3 Samplg For Proportos ad Percetages I may stuatos, the characterstc uder study o whch the observatos are collected are qualtatve ature For example, the resposes of customers may marketg surveys
More informationQualifying Exam Statistical Theory Problem Solutions August 2005
Qualfyg Exam Statstcal Theory Problem Solutos August 5. Let X, X,..., X be d uform U(,),
More informationbest estimate (mean) for X uncertainty or error in the measurement (systematic, random or statistical) best
Error Aalyss Preamble Wheever a measuremet s made, the result followg from that measuremet s always subject to ucertaty The ucertaty ca be reduced by makg several measuremets of the same quatty or by mprovg
More informationCS286.2 Lecture 4: Dinur s Proof of the PCP Theorem
CS86. Lecture 4: Dur s Proof of the PCP Theorem Scrbe: Thom Bohdaowcz Prevously, we have prove a weak verso of the PCP theorem: NP PCP 1,1/ (r = poly, q = O(1)). Wth ths result we have the desred costat
More informationHomework 1: Solutions Sid Banerjee Problem 1: (Practice with Asymptotic Notation) ORIE 4520: Stochastics at Scale Fall 2015
Fall 05 Homework : Solutos Problem : (Practce wth Asymptotc Notato) A essetal requremet for uderstadg scalg behavor s comfort wth asymptotc (or bg-o ) otato. I ths problem, you wll prove some basc facts
More informationLecture 3 Probability review (cont d)
STATS 00: Itroducto to Statstcal Iferece Autum 06 Lecture 3 Probablty revew (cot d) 3. Jot dstrbutos If radom varables X,..., X k are depedet, the ther dstrbuto may be specfed by specfyg the dvdual dstrbuto
More informationThe Mathematical Appendix
The Mathematcal Appedx Defto A: If ( Λ, Ω, where ( λ λ λ whch the probablty dstrbutos,,..., Defto A. uppose that ( Λ,,..., s a expermet type, the σ-algebra o λ λ λ are defed s deoted by ( (,,...,, σ Ω.
More informationExtreme Value Theory: An Introduction
(correcto d Extreme Value Theory: A Itroducto by Laures de Haa ad Aa Ferrera Wth ths webpage the authors ted to form the readers of errors or mstakes foud the book after publcato. We also gve extesos for
More information1 Mixed Quantum State. 2 Density Matrix. CS Density Matrices, von Neumann Entropy 3/7/07 Spring 2007 Lecture 13. ψ = α x x. ρ = p i ψ i ψ i.
CS 94- Desty Matrces, vo Neuma Etropy 3/7/07 Sprg 007 Lecture 3 I ths lecture, we wll dscuss the bascs of quatum formato theory I partcular, we wll dscuss mxed quatum states, desty matrces, vo Neuma etropy
More information( ) = ( ) ( ) Chapter 13 Asymptotic Theory and Stochastic Regressors. Stochastic regressors model
Chapter 3 Asmptotc Theor ad Stochastc Regressors The ature of eplaator varable s assumed to be o-stochastc or fed repeated samples a regresso aalss Such a assumpto s approprate for those epermets whch
More informationSTATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. x, where. = y - ˆ " 1
STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS Recall Assumpto E(Y x) η 0 + η x (lear codtoal mea fucto) Data (x, y ), (x 2, y 2 ),, (x, y ) Least squares estmator ˆ E (Y x) ˆ " 0 + ˆ " x, where ˆ
More information22 Nonparametric Methods.
22 oparametrc Methods. I parametrc models oe assumes apror that the dstrbutos have a specfc form wth oe or more ukow parameters ad oe tres to fd the best or atleast reasoably effcet procedures that aswer
More informationOrdinary Least Squares Regression. Simple Regression. Algebra and Assumptions.
Ordary Least Squares egresso. Smple egresso. Algebra ad Assumptos. I ths part of the course we are gog to study a techque for aalysg the lear relatoshp betwee two varables Y ad X. We have pars of observatos
More informationGeneral Method for Calculating Chemical Equilibrium Composition
AE 6766/Setzma Sprg 004 Geeral Metod for Calculatg Cemcal Equlbrum Composto For gve tal codtos (e.g., for gve reactats, coose te speces to be cluded te products. As a example, for combusto of ydroge wt
More informationRademacher Complexity. Examples
Algorthmc Foudatos of Learg Lecture 3 Rademacher Complexty. Examples Lecturer: Patrck Rebesch Verso: October 16th 018 3.1 Itroducto I the last lecture we troduced the oto of Rademacher complexty ad showed
More informationChapter 2 - Free Vibration of Multi-Degree-of-Freedom Systems - II
CEE49b Chapter - Free Vbrato of Mult-Degree-of-Freedom Systems - II We ca obta a approxmate soluto to the fudametal atural frequecy through a approxmate formula developed usg eergy prcples by Lord Raylegh
More informationFunctions of Random Variables
Fuctos of Radom Varables Chapter Fve Fuctos of Radom Varables 5. Itroducto A geeral egeerg aalyss model s show Fg. 5.. The model output (respose) cotas the performaces of a system or product, such as weght,
More informationCHAPTER 4 RADICAL EXPRESSIONS
6 CHAPTER RADICAL EXPRESSIONS. The th Root of a Real Number A real umber a s called the th root of a real umber b f Thus, for example: s a square root of sce. s also a square root of sce ( ). s a cube
More informationComplete Convergence and Some Maximal Inequalities for Weighted Sums of Random Variables
Joural of Sceces, Islamc Republc of Ira 8(4): -6 (007) Uversty of Tehra, ISSN 06-04 http://sceces.ut.ac.r Complete Covergece ad Some Maxmal Iequaltes for Weghted Sums of Radom Varables M. Am,,* H.R. Nl
More informationPoint Estimation: definition of estimators
Pot Estmato: defto of estmators Pot estmator: ay fucto W (X,..., X ) of a data sample. The exercse of pot estmato s to use partcular fuctos of the data order to estmate certa ukow populato parameters.
More informationDiscrete Mathematics and Probability Theory Fall 2016 Seshia and Walrand DIS 10b
CS 70 Dscrete Mathematcs ad Probablty Theory Fall 206 Sesha ad Walrad DIS 0b. Wll I Get My Package? Seaky delvery guy of some compay s out delverg packages to customers. Not oly does he had a radom package
More informationRandom Variables and Probability Distributions
Radom Varables ad Probablty Dstrbutos * If X : S R s a dscrete radom varable wth rage {x, x, x 3,. } the r = P (X = xr ) = * Let X : S R be a dscrete radom varable wth rage {x, x, x 3,.}.If x r P(X = x
More informationLecture 9: Tolerant Testing
Lecture 9: Tolerat Testg Dael Kae Scrbe: Sakeerth Rao Aprl 4, 07 Abstract I ths lecture we prove a quas lear lower boud o the umber of samples eeded to do tolerat testg for L dstace. Tolerat Testg We have
More information1 Lyapunov Stability Theory
Lyapuov Stablty heory I ths secto we cosder proofs of stablty of equlbra of autoomous systems. hs s stadard theory for olear systems, ad oe of the most mportat tools the aalyss of olear systems. It may
More informationDr. Shalabh. Indian Institute of Technology Kanpur
Aalyss of Varace ad Desg of Expermets-I MODULE -I LECTURE - SOME RESULTS ON LINEAR ALGEBRA, MATRIX THEORY AND DISTRIBUTIONS Dr. Shalabh Departmet t of Mathematcs t ad Statstcs t t Ida Isttute of Techology
More informationThe Occupancy and Coupon Collector problems
Chapter 4 The Occupacy ad Coupo Collector problems By Sarel Har-Peled, Jauary 9, 08 4 Prelmares [ Defto 4 Varace ad Stadard Devato For a radom varable X, let V E [ X [ µ X deote the varace of X, where
More informationIntroduction to Probability
Itroducto to Probablty Nader H Bshouty Departmet of Computer Scece Techo 32000 Israel e-mal: bshouty@cstechoacl 1 Combatorcs 11 Smple Rules I Combatorcs The rule of sum says that the umber of ways to choose
More informationAN UPPER BOUND FOR THE PERMANENT VERSUS DETERMINANT PROBLEM BRUNO GRENET
AN UPPER BOUND FOR THE PERMANENT VERSUS DETERMINANT PROBLEM BRUNO GRENET Abstract. The Permaet versus Determat problem s the followg: Gve a matrx X of determates over a feld of characterstc dfferet from
More information1 Solution to Problem 6.40
1 Soluto to Problem 6.40 (a We wll wrte T τ (X 1,...,X where the X s are..d. wth PDF f(x µ, σ 1 ( x µ σ g, σ where the locato parameter µ s ay real umber ad the scale parameter σ s > 0. Lettg Z X µ σ we
More informationSpecial Instructions / Useful Data
JAM 6 Set of all real umbers P A..d. B, p Posso Specal Istructos / Useful Data x,, :,,, x x Probablty of a evet A Idepedetly ad detcally dstrbuted Bomal dstrbuto wth parameters ad p Posso dstrbuto wth
More informationUNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Exam: ECON430 Statstcs Date of exam: Frday, December 8, 07 Grades are gve: Jauary 4, 08 Tme for exam: 0900 am 00 oo The problem set covers 5 pages Resources allowed:
More informationD KL (P Q) := p i ln p i q i
Cheroff-Bouds 1 The Geeral Boud Let P 1,, m ) ad Q q 1,, q m ) be two dstrbutos o m elemets, e,, q 0, for 1,, m, ad m 1 m 1 q 1 The Kullback-Lebler dvergece or relatve etroy of P ad Q s defed as m D KL
More informationThe number of observed cases The number of parameters. ith case of the dichotomous dependent variable. the ith case of the jth parameter
LOGISTIC REGRESSION Notato Model Logstc regresso regresses a dchotomous depedet varable o a set of depedet varables. Several methods are mplemeted for selectg the depedet varables. The followg otato s
More informationParameter, Statistic and Random Samples
Parameter, Statstc ad Radom Samples A parameter s a umber that descrbes the populato. It s a fxed umber, but practce we do ot kow ts value. A statstc s a fucto of the sample data,.e., t s a quatty whose
More informationLecture 3: Randomly Stopped Processes
STAT 498: Selected Topcs Stochastc Processes Wter 2018 Lecture 3: Radomly Stopped Processes Lecturer: Fag Ha Ja 29 Dsclamer: These otes have ot bee subjected to the usual scruty reserved for formal publcatos.
More informationLecture Notes Types of economic variables
Lecture Notes 3 1. Types of ecoomc varables () Cotuous varable takes o a cotuum the sample space, such as all pots o a le or all real umbers Example: GDP, Polluto cocetrato, etc. () Dscrete varables fte
More information18.413: Error Correcting Codes Lab March 2, Lecture 8
18.413: Error Correctg Codes Lab March 2, 2004 Lecturer: Dael A. Spelma Lecture 8 8.1 Vector Spaces A set C {0, 1} s a vector space f for x all C ad y C, x + y C, where we take addto to be compoet wse
More informationChain Rules for Entropy
Cha Rules for Etroy The etroy of a collecto of radom varables s the sum of codtoal etroes. Theorem: Let be radom varables havg the mass robablty x x.x. The...... The roof s obtaed by reeatg the alcato
More informationChapter 14 Logistic Regression Models
Chapter 4 Logstc Regresso Models I the lear regresso model X β + ε, there are two types of varables explaatory varables X, X,, X k ad study varable y These varables ca be measured o a cotuous scale as
More informationSampling Theory MODULE X LECTURE - 35 TWO STAGE SAMPLING (SUB SAMPLING)
Samplg Theory ODULE X LECTURE - 35 TWO STAGE SAPLIG (SUB SAPLIG) DR SHALABH DEPARTET OF ATHEATICS AD STATISTICS IDIA ISTITUTE OF TECHOLOG KAPUR Two stage samplg wth uequal frst stage uts: Cosder two stage
More informationSTK4011 and STK9011 Autumn 2016
STK4 ad STK9 Autum 6 Pot estmato Covers (most of the followg materal from chapter 7: Secto 7.: pages 3-3 Secto 7..: pages 3-33 Secto 7..: pages 35-3 Secto 7..3: pages 34-35 Secto 7.3.: pages 33-33 Secto
More informationClass 13,14 June 17, 19, 2015
Class 3,4 Jue 7, 9, 05 Pla for Class3,4:. Samplg dstrbuto of sample mea. The Cetral Lmt Theorem (CLT). Cofdece terval for ukow mea.. Samplg Dstrbuto for Sample mea. Methods used are based o CLT ( Cetral
More informationD. VQ WITH 1ST-ORDER LOSSLESS CODING
VARIABLE-RATE VQ (AKA VQ WITH ENTROPY CODING) Varable-Rate VQ = Quatzato + Lossless Varable-Legth Bary Codg A rage of optos -- from smple to complex A. Uform scalar quatzato wth varable-legth codg, oe
More informationMOLECULAR VIBRATIONS
MOLECULAR VIBRATIONS Here we wsh to vestgate molecular vbratos ad draw a smlarty betwee the theory of molecular vbratos ad Hückel theory. 1. Smple Harmoc Oscllator Recall that the eergy of a oe-dmesoal
More informationLecture 3. Sampling, sampling distributions, and parameter estimation
Lecture 3 Samplg, samplg dstrbutos, ad parameter estmato Samplg Defto Populato s defed as the collecto of all the possble observatos of terest. The collecto of observatos we take from the populato s called
More informationTHE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA
THE ROYAL STATISTICAL SOCIETY 3 EXAMINATIONS SOLUTIONS GRADUATE DIPLOMA PAPER I STATISTICAL THEORY & METHODS The Socety provdes these solutos to assst caddates preparg for the examatos future years ad
More informationChapter 4 (Part 1): Non-Parametric Classification (Sections ) Pattern Classification 4.3) Announcements
Aoucemets No-Parametrc Desty Estmato Techques HW assged Most of ths lecture was o the blacboard. These sldes cover the same materal as preseted DHS Bometrcs CSE 90-a Lecture 7 CSE90a Fall 06 CSE90a Fall
More informationChapter 8. Inferences about More Than Two Population Central Values
Chapter 8. Ifereces about More Tha Two Populato Cetral Values Case tudy: Effect of Tmg of the Treatmet of Port-We tas wth Lasers ) To vestgate whether treatmet at a youg age would yeld better results tha
More informationVARIABLE-RATE VQ (AKA VQ WITH ENTROPY CODING)
VARIABLE-RATE VQ (AKA VQ WITH ENTROPY CODING) Varable-Rate VQ = Quatzato + Lossless Varable-Legth Bary Codg A rage of optos -- from smple to complex a. Uform scalar quatzato wth varable-legth codg, oe
More informationCIS 800/002 The Algorithmic Foundations of Data Privacy October 13, Lecture 9. Database Update Algorithms: Multiplicative Weights
CIS 800/002 The Algorthmc Foudatos of Data Prvacy October 13, 2011 Lecturer: Aaro Roth Lecture 9 Scrbe: Aaro Roth Database Update Algorthms: Multplcatve Weghts We ll recall aga) some deftos from last tme:
More informationSimulation Output Analysis
Smulato Output Aalyss Summary Examples Parameter Estmato Sample Mea ad Varace Pot ad Iterval Estmato ermatg ad o-ermatg Smulato Mea Square Errors Example: Sgle Server Queueg System x(t) S 4 S 4 S 3 S 5
More informationChapter 4 Multiple Random Variables
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 Revew for Chapter 4-5 Notes: Although all deftos ad theorems troduced our lectures ad ths ote are mportat ad you should be famlar wth, but I put those
More informationLINEAR RECURRENT SEQUENCES AND POWERS OF A SQUARE MATRIX
INTEGERS: ELECTRONIC JOURNAL OF COMBINATORIAL NUMBER THEORY 6 2006, #A12 LINEAR RECURRENT SEQUENCES AND POWERS OF A SQUARE MATRIX Hacèe Belbachr 1 USTHB, Departmet of Mathematcs, POBox 32 El Ala, 16111,
More informationIntroduction to local (nonparametric) density estimation. methods
Itroducto to local (oparametrc) desty estmato methods A slecture by Yu Lu for ECE 66 Sprg 014 1. Itroducto Ths slecture troduces two local desty estmato methods whch are Parze desty estmato ad k-earest
More information5 Short Proofs of Simplified Stirling s Approximation
5 Short Proofs of Smplfed Strlg s Approxmato Ofr Gorodetsky, drtymaths.wordpress.com Jue, 20 0 Itroducto Strlg s approxmato s the followg (somewhat surprsg) approxmato of the factoral,, usg elemetary fuctos:
More informationRandom Variables. ECE 313 Probability with Engineering Applications Lecture 8 Professor Ravi K. Iyer University of Illinois
Radom Varables ECE 313 Probablty wth Egeerg Alcatos Lecture 8 Professor Rav K. Iyer Uversty of Illos Iyer - Lecture 8 ECE 313 Fall 013 Today s Tocs Revew o Radom Varables Cumulatve Dstrbuto Fucto (CDF
More informationContinuous Distributions
7//3 Cotuous Dstrbutos Radom Varables of the Cotuous Type Desty Curve Percet Desty fucto, f (x) A smooth curve that ft the dstrbuto 3 4 5 6 7 8 9 Test scores Desty Curve Percet Probablty Desty Fucto, f
More informationLecture Notes to Rice Chapter 5
ECON 430 Revsed Sept. 06 Lecture Notes to Rce Chapter 5 By H. Goldste. Chapter 5 gves a troducto to probablstc approxmato methods, but s suffcet for the eeds of a adequate study of ecoometrcs. The commo
More information2. Independence and Bernoulli Trials
. Ideedece ad Beroull Trals Ideedece: Evets ad B are deedet f B B. - It s easy to show that, B deedet mles, B;, B are all deedet ars. For examle, ad so that B or B B B B B φ,.e., ad B are deedet evets.,
More information6.867 Machine Learning
6.867 Mache Learg Problem set Due Frday, September 9, rectato Please address all questos ad commets about ths problem set to 6.867-staff@a.mt.edu. You do ot eed to use MATLAB for ths problem set though
More informationChapter 13, Part A Analysis of Variance and Experimental Design. Introduction to Analysis of Variance. Introduction to Analysis of Variance
Chapter, Part A Aalyss of Varace ad Epermetal Desg Itroducto to Aalyss of Varace Aalyss of Varace: Testg for the Equalty of Populato Meas Multple Comparso Procedures Itroducto to Aalyss of Varace Aalyss
More informationNonlinear Piecewise-Defined Difference Equations with Reciprocal Quadratic Terms
Joural of Matematcs ad Statstcs Orgal Researc Paper Nolear Pecewse-Defed Dfferece Equatos wt Recprocal Quadratc Terms Ramada Sabra ad Saleem Safq Al-Asab Departmet of Matematcs, Faculty of Scece, Jaza
More informationLarge and Moderate Deviation Principles for Kernel Distribution Estimator
Iteratoal Mathematcal Forum, Vol. 9, 2014, o. 18, 871-890 HIKARI Ltd, www.m-hkar.com http://dx.do.org/10.12988/mf.2014.4488 Large ad Moderate Devato Prcples for Kerel Dstrbuto Estmator Yousr Slaou Uversté
More informationNotes on Censored EL, and Harzard
Notes o Cesored EL, ad Harzard Ma Zhou I survval aalyss, the statstcs volvg the hazard fuctos are usually easer to hadle mathematcally the those volvg the dstrbutos. For example, t s easer to show the
More informationIntroduction to Matrices and Matrix Approach to Simple Linear Regression
Itroducto to Matrces ad Matrx Approach to Smple Lear Regresso Matrces Defto: A matrx s a rectagular array of umbers or symbolc elemets I may applcatos, the rows of a matrx wll represet dvduals cases (people,
More informationAnswer key to problem set # 2 ECON 342 J. Marcelo Ochoa Spring, 2009
Aswer key to problem set # ECON 34 J. Marcelo Ochoa Sprg, 009 Problem. For T cosder the stadard pael data model: y t x t β + α + ǫ t a Numercally compare the fxed effect ad frst dfferece estmates. b Compare
More information(b) By independence, the probability that the string 1011 is received correctly is
Soluto to Problem 1.31. (a) Let A be the evet that a 0 s trasmtted. Usg the total probablty theorem, the desred probablty s P(A)(1 ɛ ( 0)+ 1 P(A) ) (1 ɛ 1)=p(1 ɛ 0)+(1 p)(1 ɛ 1). (b) By depedece, the probablty
More informationFeature Selection: Part 2. 1 Greedy Algorithms (continued from the last lecture)
CSE 546: Mache Learg Lecture 6 Feature Selecto: Part 2 Istructor: Sham Kakade Greedy Algorthms (cotued from the last lecture) There are varety of greedy algorthms ad umerous amg covetos for these algorthms.
More informationSTRONG CONSISTENCY OF LEAST SQUARES ESTIMATE IN MULTIPLE REGRESSION WHEN THE ERROR VARIANCE IS INFINITE
Statstca Sca 9(1999), 289-296 STRONG CONSISTENCY OF LEAST SQUARES ESTIMATE IN MULTIPLE REGRESSION WHEN THE ERROR VARIANCE IS INFINITE J Mgzhog ad Che Xru GuZhou Natoal College ad Graduate School, Chese
More informationMA 524 Homework 6 Solutions
MA 524 Homework 6 Solutos. Sce S(, s the umber of ways to partto [] to k oempty blocks, ad c(, s the umber of ways to partto to k oempty blocks ad also the arrage each block to a cycle, we must have S(,
More informationMultivariate Transformation of Variables and Maximum Likelihood Estimation
Marquette Uversty Multvarate Trasformato of Varables ad Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Assocate Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 03 by Marquette Uversty
More informationESS Line Fitting
ESS 5 014 17. Le Fttg A very commo problem data aalyss s lookg for relatoshpetwee dfferet parameters ad fttg les or surfaces to data. The smplest example s fttg a straght le ad we wll dscuss that here
More information3. Basic Concepts: Consequences and Properties
: 3. Basc Cocepts: Cosequeces ad Propertes Markku Jutt Overvew More advaced cosequeces ad propertes of the basc cocepts troduced the prevous lecture are derved. Source The materal s maly based o Sectos.6.8
More informationAssignment 5/MATH 247/Winter Due: Friday, February 19 in class (!) (answers will be posted right after class)
Assgmet 5/MATH 7/Wter 00 Due: Frday, February 9 class (!) (aswers wll be posted rght after class) As usual, there are peces of text, before the questos [], [], themselves. Recall: For the quadratc form
More informationNon-uniform Turán-type problems
Joural of Combatoral Theory, Seres A 111 2005 106 110 wwwelsevercomlocatecta No-uform Turá-type problems DhruvMubay 1, Y Zhao 2 Departmet of Mathematcs, Statstcs, ad Computer Scece, Uversty of Illos at
More information