CS286.2 Lecture 4: Dinur s Proof of the PCP Theorem
|
|
- Norma Cross
- 5 years ago
- Views:
Transcription
1 CS86. Lecture 4: Dur s Proof of the PCP Theorem Scrbe: Thom Bohdaowcz Prevously, we have prove a weak verso of the PCP theorem: NP PCP 1,1/ (r = poly, q = O(1)). Wth ths result we have the desred costat completeess ad soudess, but the proof s expoetally log. It takes dffcult work to shorte the proof to get the full PCP theorem: NP PCP 1,1/ (r = O(log ), q = O(1)). Now we wll see a dfferet approach to the proof of the PCP theorem whch was recetly dscovered (006) by Irt Dur. The approach here wll be to take a result wth a small soudess gap, NP PCP 1,1 1 m (r = O(log ), q = O(1)), ad gradually amplfy the soudess to get the desred gap whle keepg the legth of the proof costat. The proof ca be uderstood usg the CSP varat of the PCP theorem: Theorem 1. For ay L NP there s a polyomal-tme mappg x {0, 1} ϕ x where ϕ x s a q-csp over m = poly() clauses, q O(1), such that: x L = ω(ϕ x ) = 1 x / L = ω(ϕ x ) 1/ The ma Lemma facltatg Dur s proof s the followg: Lemma. There exsts a effcet polyomal-tme procedure mappg a -CSP ϕ = ϕ () to -CSP ϕ = ϕ (+1) as follows ϕ = ϕ () over Σ = {0, 1} = ϕ = ϕ (+1) over Σ = {0, 1} ϕ () has m costrats = ϕ (+1) has m Cm costrats where C s some uversal costat ω(ϕ () ) = 1 = ω(ϕ (+1) ) = 1 ω(ϕ () ) 1 ɛ = ω(ϕ (+1) ) 1 ɛ To see why ths lemma mples the PCP theorem, here s the dea: take some NP-complete laguage lke 3 COLORING. By defto t s NP-hard to decde for the correspodg - CSP ϕ (0) whether ω(ϕ (0) ) = 1 or ω(ϕ (0) ) 1 m 1, where m s the umber of costrats (whch the case of 3 COLORING s the umber of edges the graph). Applyg the above Lemma oce brgs us from {ω(ϕ (0) ) = 1 or ω(ϕ (0) ) 1 m 1 } to {ω(ϕ(1) ) = 1 or ω(ϕ (1) ) 1 m }, 1
2 ad so applyg t t tmes brgs us to {ω(ϕ (t) ) = 1 or ω(ϕ (t) ) 1 t m }. Settg t log m, we re doe, otg that ϕ (t) C t m m log C+1 remas polyomal the sze of the orgal stace ϕ (0). The proof of the Lemma has two ma steps: gap amplfcato (GA) ad alphabet reducto (AR). These steps operate as follows o our orgal -CSP stace: CSP: ϕ = ϕ () a -CSP GA ϕ = ϕ () a -CSP AR ϕ = ϕ (+1) a -CSP Clauses: m GA m Cm AR m C m Alphabet: Σ = {0, 1} GA Σ = [w] = {0, 1,..., w 1} where w s some larger teger AR Σ = {0, 1} Completeess: ω(ϕ = ϕ () ) = 1 GA ω(ϕ = ϕ () ) = 1 AR ω(ϕ = ϕ (+1) ) = 1 Soudess: ω(ϕ = ϕ () ) 1 ɛ GA ω(ϕ = ϕ () ) 1 6ɛ AR ω(ϕ = ϕ (+1) ) 1 ɛ As oted above, repeatg these steps O(log m) tmes gves a costat gap wth oly a polyomal blow-up the umber of costrats (sce all clauses volve a costat umber of varables, the same automatcally holds for the umber of varables). The harder part s costructg the gap amplfcato procedure, so that s what we wll look at ths lecture. It requres the use of expader graphs. Expader Graphs A expader graph s a graph G = (V, E) whch has good expaso propertes. A loose defto would be that there s ever a small subset of vertces that has a lot of edges sde, but coectg the subset to the rest of the graph wth oly a few edges. We focus o -vertex d-regular graphs, ad gve the followg defto. Defto 3. Let, d be tegers ad 0 ρ < 1. A -vertex d-regular graph G s called a (, d, ρ)-expader f S [] s.t. S, E(S, S) ρd S. (1) Note that d S s the maxmum umber of edges that we could talk about. Also, otce that we always have ρ 1.1 ). The followg theorem states the exstece of good expaders. Theorem 4. ɛ > 0, d = d(ɛ), N = N(ɛ) s.t. N, G whch s a (, d, 1 ɛ)- expader. 1 To see ths, for ay graph G cosder a radomly chose S obtaed by cludg every vertex depedetly wth probablty 1/. I expectato the set S wll have exactly d/4 edges gog out, ad so there must exst a set S havg as may edges leavg t.
3 We wll ot prove t, but t s ot too hard to show usg the probablstc method: a radom graph works. Explct costructos, o the other had, are much harder. The followg lemma about expaders states the key property that wll be used the aalyss of Dur s gap amplfcato procedure. Lemma 5. Suppose that G s a (, d, ρ)-expader. The for ay set S [] wth S, t, Pr u t v(u S, v S) S ( ( S 1 ρ + ) t ) () where u t v deotes choosg a uformly radom u V, makg a t-step radom walk startg there, ad arrvg at v V. Proof. We ll prove t for t = 1. By defto of expaso, Pr u t v(u S, v S) = Pr(u S)Pr(v S u S) = S S S (1 ρ) ( S + 1 ) ρ ( S + 1 ρ where the frst equalty uses S / 1/ ad the last that 1 ρ 1 ρ for ρ. Note ths s a bt of a wasteful boud. Alteratvely there s a algebrac techque: ρ s related to the secod largest egevalue λ (G) of the adjacecy matrx A(G) as λ (G) 1 ρ (ths s called Cheeger s equalty). The use λ (G t ) = λ (G) t ad the use ρ 1 λ. G t s a graph whose adjacecy matrx specfes the paths of legth t G as edges, rather tha the paths of legth 1; applyg the t = 1 case prove above to G t would yeld the lemma. We omt the detals. Proof/Costructo of Gap Amplfcato Step We just wat to get the ma dea of ths proof, ad so we assume that the costrat graph G of ϕ s a (, d, ρ)-expader for some ρ < 1. (A addtoal step Dur s proof shows that ths ca always be the case, wthout loss of geeralty.) Suppose ω(ϕ) 1 ɛ. The ay assgmet to the vertces volates a fracto of at least ɛ of the costrats. These correspod to a bad set of edges of G, whch we ca call S. We wat to fgure out how to defe ϕ as specfed for the gap amplfcato step. Fx a parameter t ad defe ϕ as follows: ϕ s varables are the same as ϕ s ϕ s alphabet s Σ = {0, 1} 1+d+d +...+d t. The dea s that each varable holds a value for tself, ts earest eghbors ad etc... all the way to the t-th earest eghbors. 3 ),
4 The costrats are weghted costrats; we ca always go back to uweghted costrats by duplcatg the heavy (hgher probablty) costrats as may tmes as requred. The costrats are geerated accordg to the followg dstrbuto: 1. Choose a radom vertex u.. Make a t-step radom walk G from u to v. 3. Look at the values of the varables of ϕ, x u ad x v, that are assocated to the vertces u ad v. Each of these varables provdes a assgmet of values {0, 1} to all vertces at dstace at most t from them. Check that the values provded by x u ad x v agree o all vertces the path u v, ad that the commo values satsfy all of ϕ s costrats alog the path. The whole checkg procedure makes a sgle costrat that s placed o a (u, v) edge for ϕ. Why does ths work? The frst thg to check would be that ω(ϕ) = 1 = ω(ϕ ) = 1, whch s clear: all varables x ϕ ca be assged values cosstet wth a sgle satsfyg assgmet x for ϕ. Secod ad more mportatly s to check that ω(ϕ) 1 ɛ = ω(ϕ ) 1 6ɛ. Let s take a assgmet y to the varables for ϕ, such that y volates a fracto (1 ωϕ ) = 1 δ of the clauses. What we ll show: there exsts some x that volates a fracto at most 6 δ of the costrats of ϕ. Sce by assumpto ω(ϕ) 1 ε, we must have 6 δ coclude the proof. ɛ ad so δ 6ɛ, whch wll We make a mportat smplfcato. Assume that y s obtaed the correct way from some x for ϕ: whe two y overlap (that s, they provde values for the same vertces of G) the ther values match. If ths smplfyg assumpto s ot true, the the proof volves a addtoal step of amoutg to a decodg procedure whch defes x at ay vertex to be the majorty value amog those provded by all eghbourg y; t s the possble to show that there s always a strog majorty. Our smplfyg assumpto thus amouts to assumg that majorty s uamty! I So, our goal s to show that δ 6ɛ. We ll show that f y volates at most a fracto δ of the clauses, the x volates at most a fracto δ t of clauses. But the best x volates at least a fracto ɛ of clauses.. so, δ t ɛ ad so δ tɛ ad we ca take t 6 to get what we wat. (Note that we could get better for larger t but remember that the alphabet sze for ϕ creases very quckly wth t...) We eed to show that a radom costrat s volated wth probablty of at least tɛ. A radom costrat s a whole buch of costrats from ϕ. If these were chose depedetly the the probablty we completely avod the bad set S would be (1 ɛ) t 1 tɛ, ad so wth probablty at least tɛ we see a volato. However, costrats are ot depedet: they are selected from the radom walk. The followg lemma wll let us complete the proof. Lemma 6. Let G be a (, d, ρ) expader. Let S be a subset of edges of G such that S /m = ε, where m s the total umber of edges. Let t < 1/(dε) be a teger. The ρ Pr (at least oe edge of the t-walk s S) u t v 8d tε. 4
5 Applyg the lemma wth S the set of edges correspodg to costrats that are ot satsfed by the assgmet x, we obta that δ ρ /(8d)tε. Choosg t = 48d/ρ (where ρ s the expaso factor of a good expader used the costructo, ad ca be take to be a small costat, e.g. ρ = 1/4) the gves us δ 6ε (as log as ε s ot too large, case we d be doe already), as desred. Proof. For = 1,..., t defe dcator varables X = 1 f the -th edge of the t-step radom walk from u to v s S, ad X = 0 otherwse. The E[X ] = S /m = ε: sce the frst vertex u of the walk s chose uformly at radom, ay dvdual edge of the walk s also a uformly radom edge G. By learty of expectato, we deduce [ E[X] = E ] X = E[X ] = tε, where X s the umber of edges alog the walk that are S. The probablty the lemma s Pr(X > 0). We wll use a secod momet boud Pr(X > 0) E[X] E[X ], (3) whch holds for ay radom varable X 0. For ths t oly remas to lower boud the secod momet E[X ]. Let S be the set of vertces G that touch a edge S. Wrte ] E[X ] = E[ X X j 1,j t = E[X X j ] 1,j t = E [ X ] + E[X X j ] = tε + Pr(-th edge S ad j-th edge S) tε + tε + tε + t (dε) + tdε/ρ (3 + 4d/ρ )tε, Pr(-th vertex S ad j-th vertex S) dε (dε + ( (1 ρ )/ ) ) j where the here the key step s gve the secod equalty, whch we appled Lemma 5 (the factor dε comes from the fact that S d S ). I the last equalty we used tdε < 1. Together wth (3) ths cocludes the proof of the lemma. 5
Lecture 9: Tolerant Testing
Lecture 9: Tolerat Testg Dael Kae Scrbe: Sakeerth Rao Aprl 4, 07 Abstract I ths lecture we prove a quas lear lower boud o the umber of samples eeded to do tolerat testg for L dstace. Tolerat Testg We have
More information8.1 Hashing Algorithms
CS787: Advaced Algorthms Scrbe: Mayak Maheshwar, Chrs Hrchs Lecturer: Shuch Chawla Topc: Hashg ad NP-Completeess Date: September 21 2007 Prevously we looked at applcatos of radomzed algorthms, ad bega
More informationDiscrete Mathematics and Probability Theory Fall 2016 Seshia and Walrand DIS 10b
CS 70 Dscrete Mathematcs ad Probablty Theory Fall 206 Sesha ad Walrad DIS 0b. Wll I Get My Package? Seaky delvery guy of some compay s out delverg packages to customers. Not oly does he had a radom package
More informationPTAS for Bin-Packing
CS 663: Patter Matchg Algorthms Scrbe: Che Jag /9/00. Itroducto PTAS for B-Packg The B-Packg problem s NP-hard. If we use approxmato algorthms, the B-Packg problem could be solved polyomal tme. For example,
More informationBounds on the expected entropy and KL-divergence of sampled multinomial distributions. Brandon C. Roy
Bouds o the expected etropy ad KL-dvergece of sampled multomal dstrbutos Brado C. Roy bcroy@meda.mt.edu Orgal: May 18, 2011 Revsed: Jue 6, 2011 Abstract Iformato theoretc quattes calculated from a sampled
More informationLecture 3 Probability review (cont d)
STATS 00: Itroducto to Statstcal Iferece Autum 06 Lecture 3 Probablty revew (cot d) 3. Jot dstrbutos If radom varables X,..., X k are depedet, the ther dstrbuto may be specfed by specfyg the dvdual dstrbuto
More informationClass 13,14 June 17, 19, 2015
Class 3,4 Jue 7, 9, 05 Pla for Class3,4:. Samplg dstrbuto of sample mea. The Cetral Lmt Theorem (CLT). Cofdece terval for ukow mea.. Samplg Dstrbuto for Sample mea. Methods used are based o CLT ( Cetral
More informationD KL (P Q) := p i ln p i q i
Cheroff-Bouds 1 The Geeral Boud Let P 1,, m ) ad Q q 1,, q m ) be two dstrbutos o m elemets, e,, q 0, for 1,, m, ad m 1 m 1 q 1 The Kullback-Lebler dvergece or relatve etroy of P ad Q s defed as m D KL
More information9 U-STATISTICS. Eh =(m!) 1 Eh(X (1),..., X (m ) ) i.i.d
9 U-STATISTICS Suppose,,..., are P P..d. wth CDF F. Our goal s to estmate the expectato t (P)=Eh(,,..., m ). Note that ths expectato requres more tha oe cotrast to E, E, or Eh( ). Oe example s E or P((,
More informationCHAPTER 4 RADICAL EXPRESSIONS
6 CHAPTER RADICAL EXPRESSIONS. The th Root of a Real Number A real umber a s called the th root of a real umber b f Thus, for example: s a square root of sce. s also a square root of sce ( ). s a cube
More informationMATH 247/Winter Notes on the adjoint and on normal operators.
MATH 47/Wter 00 Notes o the adjot ad o ormal operators I these otes, V s a fte dmesoal er product space over, wth gve er * product uv, T, S, T, are lear operators o V U, W are subspaces of V Whe we say
More informationEconometric Methods. Review of Estimation
Ecoometrc Methods Revew of Estmato Estmatg the populato mea Radom samplg Pot ad terval estmators Lear estmators Ubased estmators Lear Ubased Estmators (LUEs) Effcecy (mmum varace) ad Best Lear Ubased Estmators
More informationChapter 9 Jordan Block Matrices
Chapter 9 Jorda Block atrces I ths chapter we wll solve the followg problem. Gve a lear operator T fd a bass R of F such that the matrx R (T) s as smple as possble. f course smple s a matter of taste.
More informationAlgorithms Design & Analysis. Hash Tables
Algorthms Desg & Aalyss Hash Tables Recap Lower boud Order statstcs 2 Today s topcs Drect-accessble table Hash tables Hash fuctos Uversal hashg Perfect Hashg Ope addressg 3 Symbol-table problem Symbol
More informationSpecial Instructions / Useful Data
JAM 6 Set of all real umbers P A..d. B, p Posso Specal Istructos / Useful Data x,, :,,, x x Probablty of a evet A Idepedetly ad detcally dstrbuted Bomal dstrbuto wth parameters ad p Posso dstrbuto wth
More informationLecture 02: Bounding tail distributions of a random variable
CSCI-B609: A Theorst s Toolkt, Fall 206 Aug 25 Lecture 02: Boudg tal dstrbutos of a radom varable Lecturer: Yua Zhou Scrbe: Yua Xe & Yua Zhou Let us cosder the ubased co flps aga. I.e. let the outcome
More informationCIS 800/002 The Algorithmic Foundations of Data Privacy October 13, Lecture 9. Database Update Algorithms: Multiplicative Weights
CIS 800/002 The Algorthmc Foudatos of Data Prvacy October 13, 2011 Lecturer: Aaro Roth Lecture 9 Scrbe: Aaro Roth Database Update Algorthms: Multplcatve Weghts We ll recall aga) some deftos from last tme:
More informationThe Occupancy and Coupon Collector problems
Chapter 4 The Occupacy ad Coupo Collector problems By Sarel Har-Peled, Jauary 9, 08 4 Prelmares [ Defto 4 Varace ad Stadard Devato For a radom varable X, let V E [ X [ µ X deote the varace of X, where
More informationHard Core Predicates: How to encrypt? Recap
Hard Core Predcates: How to ecrypt? Debdeep Mukhopadhyay IIT Kharagpur Recap A ecrypto scheme s secured f for every probablstc adversary A carryg out some specfed kd of attack ad for every polyomal p(.),
More informationChapter 5 Properties of a Random Sample
Lecture 6 o BST 63: Statstcal Theory I Ku Zhag, /0/008 Revew for the prevous lecture Cocepts: t-dstrbuto, F-dstrbuto Theorems: Dstrbutos of sample mea ad sample varace, relatoshp betwee sample mea ad sample
More informationUNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Postpoed exam: ECON430 Statstcs Date of exam: Jauary 0, 0 Tme for exam: 09:00 a.m. :00 oo The problem set covers 5 pages Resources allowed: All wrtte ad prted
More information{ }{ ( )} (, ) = ( ) ( ) ( ) Chapter 14 Exercises in Sampling Theory. Exercise 1 (Simple random sampling): Solution:
Chapter 4 Exercses Samplg Theory Exercse (Smple radom samplg: Let there be two correlated radom varables X ad A sample of sze s draw from a populato by smple radom samplg wthout replacemet The observed
More informationSimulation Output Analysis
Smulato Output Aalyss Summary Examples Parameter Estmato Sample Mea ad Varace Pot ad Iterval Estmato ermatg ad o-ermatg Smulato Mea Square Errors Example: Sgle Server Queueg System x(t) S 4 S 4 S 3 S 5
More informationLecture 4 Sep 9, 2015
CS 388R: Radomzed Algorthms Fall 205 Prof. Erc Prce Lecture 4 Sep 9, 205 Scrbe: Xagru Huag & Chad Voegele Overvew I prevous lectures, we troduced some basc probablty, the Cheroff boud, the coupo collector
More informationOrdinary Least Squares Regression. Simple Regression. Algebra and Assumptions.
Ordary Least Squares egresso. Smple egresso. Algebra ad Assumptos. I ths part of the course we are gog to study a techque for aalysg the lear relatoshp betwee two varables Y ad X. We have pars of observatos
More informationHomework 1: Solutions Sid Banerjee Problem 1: (Practice with Asymptotic Notation) ORIE 4520: Stochastics at Scale Fall 2015
Fall 05 Homework : Solutos Problem : (Practce wth Asymptotc Notato) A essetal requremet for uderstadg scalg behavor s comfort wth asymptotc (or bg-o ) otato. I ths problem, you wll prove some basc facts
More informationA tighter lower bound on the circuit size of the hardest Boolean functions
Electroc Colloquum o Computatoal Complexty, Report No. 86 2011) A tghter lower boud o the crcut sze of the hardest Boolea fuctos Masak Yamamoto Abstract I [IPL2005], Fradse ad Mlterse mproved bouds o the
More informationLecture 1. (Part II) The number of ways of partitioning n distinct objects into k distinct groups containing n 1,
Lecture (Part II) Materals Covered Ths Lecture: Chapter 2 (2.6 --- 2.0) The umber of ways of parttog dstct obects to dstct groups cotag, 2,, obects, respectvely, where each obect appears exactly oe group
More informationChapter 4 (Part 1): Non-Parametric Classification (Sections ) Pattern Classification 4.3) Announcements
Aoucemets No-Parametrc Desty Estmato Techques HW assged Most of ths lecture was o the blacboard. These sldes cover the same materal as preseted DHS Bometrcs CSE 90-a Lecture 7 CSE90a Fall 06 CSE90a Fall
More informationSummary of the lecture in Biostatistics
Summary of the lecture Bostatstcs Probablty Desty Fucto For a cotuos radom varable, a probablty desty fucto s a fucto such that: 0 dx a b) b a dx A probablty desty fucto provdes a smple descrpto of the
More informationPseudo-random Functions
Pseudo-radom Fuctos Debdeep Mukhopadhyay IIT Kharagpur We have see the costructo of PRG (pseudo-radom geerators) beg costructed from ay oe-way fuctos. Now we shall cosder a related cocept: Pseudo-radom
More informationIntroduction to local (nonparametric) density estimation. methods
Itroducto to local (oparametrc) desty estmato methods A slecture by Yu Lu for ECE 66 Sprg 014 1. Itroducto Ths slecture troduces two local desty estmato methods whch are Parze desty estmato ad k-earest
More information18.413: Error Correcting Codes Lab March 2, Lecture 8
18.413: Error Correctg Codes Lab March 2, 2004 Lecturer: Dael A. Spelma Lecture 8 8.1 Vector Spaces A set C {0, 1} s a vector space f for x all C ad y C, x + y C, where we take addto to be compoet wse
More informationStrong Convergence of Weighted Averaged Approximants of Asymptotically Nonexpansive Mappings in Banach Spaces without Uniform Convexity
BULLETIN of the MALAYSIAN MATHEMATICAL SCIENCES SOCIETY Bull. Malays. Math. Sc. Soc. () 7 (004), 5 35 Strog Covergece of Weghted Averaged Appromats of Asymptotcally Noepasve Mappgs Baach Spaces wthout
More informationDimensionality Reduction and Learning
CMSC 35900 (Sprg 009) Large Scale Learg Lecture: 3 Dmesoalty Reducto ad Learg Istructors: Sham Kakade ad Greg Shakharovch L Supervsed Methods ad Dmesoalty Reducto The theme of these two lectures s that
More informationChapter 14 Logistic Regression Models
Chapter 4 Logstc Regresso Models I the lear regresso model X β + ε, there are two types of varables explaatory varables X, X,, X k ad study varable y These varables ca be measured o a cotuous scale as
More informationSTATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. x, where. = y - ˆ " 1
STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS Recall Assumpto E(Y x) η 0 + η x (lear codtoal mea fucto) Data (x, y ), (x 2, y 2 ),, (x, y ) Least squares estmator ˆ E (Y x) ˆ " 0 + ˆ " x, where ˆ
More informationX X X E[ ] E X E X. is the ()m n where the ( i,)th. j element is the mean of the ( i,)th., then
Secto 5 Vectors of Radom Varables Whe workg wth several radom varables,,..., to arrage them vector form x, t s ofte coveet We ca the make use of matrx algebra to help us orgaze ad mapulate large umbers
More informationX ε ) = 0, or equivalently, lim
Revew for the prevous lecture Cocepts: order statstcs Theorems: Dstrbutos of order statstcs Examples: How to get the dstrbuto of order statstcs Chapter 5 Propertes of a Radom Sample Secto 55 Covergece
More informationChapter 4 Multiple Random Variables
Revew for the prevous lecture: Theorems ad Examples: How to obta the pmf (pdf) of U = g (, Y) ad V = g (, Y) Chapter 4 Multple Radom Varables Chapter 44 Herarchcal Models ad Mxture Dstrbutos Examples:
More informationAN UPPER BOUND FOR THE PERMANENT VERSUS DETERMINANT PROBLEM BRUNO GRENET
AN UPPER BOUND FOR THE PERMANENT VERSUS DETERMINANT PROBLEM BRUNO GRENET Abstract. The Permaet versus Determat problem s the followg: Gve a matrx X of determates over a feld of characterstc dfferet from
More informationρ < 1 be five real numbers. The
Lecture o BST 63: Statstcal Theory I Ku Zhag, /0/006 Revew for the prevous lecture Deftos: covarace, correlato Examples: How to calculate covarace ad correlato Theorems: propertes of correlato ad covarace
More informationECONOMETRIC THEORY. MODULE VIII Lecture - 26 Heteroskedasticity
ECONOMETRIC THEORY MODULE VIII Lecture - 6 Heteroskedastcty Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur . Breusch Paga test Ths test ca be appled whe the replcated data
More informationLecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model
Lecture 7. Cofdece Itervals ad Hypothess Tests the Smple CLR Model I lecture 6 we troduced the Classcal Lear Regresso (CLR) model that s the radom expermet of whch the data Y,,, K, are the outcomes. The
More informationFeature Selection: Part 2. 1 Greedy Algorithms (continued from the last lecture)
CSE 546: Mache Learg Lecture 6 Feature Selecto: Part 2 Istructor: Sham Kakade Greedy Algorthms (cotued from the last lecture) There are varety of greedy algorthms ad umerous amg covetos for these algorthms.
More informationThe Mathematical Appendix
The Mathematcal Appedx Defto A: If ( Λ, Ω, where ( λ λ λ whch the probablty dstrbutos,,..., Defto A. uppose that ( Λ,,..., s a expermet type, the σ-algebra o λ λ λ are defed s deoted by ( (,,...,, σ Ω.
More informationPoint Estimation: definition of estimators
Pot Estmato: defto of estmators Pot estmator: ay fucto W (X,..., X ) of a data sample. The exercse of pot estmato s to use partcular fuctos of the data order to estmate certa ukow populato parameters.
More informationRademacher Complexity. Examples
Algorthmc Foudatos of Learg Lecture 3 Rademacher Complexty. Examples Lecturer: Patrck Rebesch Verso: October 16th 018 3.1 Itroducto I the last lecture we troduced the oto of Rademacher complexty ad showed
More informationå 1 13 Practice Final Examination Solutions - = CS109 Dec 5, 2018
Chrs Pech Fal Practce CS09 Dec 5, 08 Practce Fal Examato Solutos. Aswer: 4/5 8/7. There are multle ways to obta ths aswer; here are two: The frst commo method s to sum over all ossbltes for the rak of
More informationExercises for Square-Congruence Modulo n ver 11
Exercses for Square-Cogruece Modulo ver Let ad ab,.. Mark True or False. a. 3S 30 b. 3S 90 c. 3S 3 d. 3S 4 e. 4S f. 5S g. 0S 55 h. 8S 57. 9S 58 j. S 76 k. 6S 304 l. 47S 5347. Fd the equvalece classes duced
More informationUNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Exam: ECON430 Statstcs Date of exam: Frday, December 8, 07 Grades are gve: Jauary 4, 08 Tme for exam: 0900 am 00 oo The problem set covers 5 pages Resources allowed:
More informationFunctions of Random Variables
Fuctos of Radom Varables Chapter Fve Fuctos of Radom Varables 5. Itroducto A geeral egeerg aalyss model s show Fg. 5.. The model output (respose) cotas the performaces of a system or product, such as weght,
More informationIntroduction to Probability
Itroducto to Probablty Nader H Bshouty Departmet of Computer Scece Techo 32000 Israel e-mal: bshouty@cstechoacl 1 Combatorcs 11 Smple Rules I Combatorcs The rule of sum says that the umber of ways to choose
More information1 Solution to Problem 6.40
1 Soluto to Problem 6.40 (a We wll wrte T τ (X 1,...,X where the X s are..d. wth PDF f(x µ, σ 1 ( x µ σ g, σ where the locato parameter µ s ay real umber ad the scale parameter σ s > 0. Lettg Z X µ σ we
More information18.657: Mathematics of Machine Learning
8.657: Mathematcs of Mache Learg Lecturer: Phlppe Rgollet Lecture 3 Scrbe: James Hrst Sep. 6, 205.5 Learg wth a fte dctoary Recall from the ed of last lecture our setup: We are workg wth a fte dctoary
More information1 Onto functions and bijections Applications to Counting
1 Oto fuctos ad bectos Applcatos to Coutg Now we move o to a ew topc. Defto 1.1 (Surecto. A fucto f : A B s sad to be surectve or oto f for each b B there s some a A so that f(a B. What are examples of
More informationBounds for the Connective Eccentric Index
It. J. Cotemp. Math. Sceces, Vol. 7, 0, o. 44, 6-66 Bouds for the Coectve Eccetrc Idex Nlaja De Departmet of Basc Scece, Humates ad Socal Scece (Mathematcs Calcutta Isttute of Egeerg ad Maagemet Kolkata,
More informationLecture 3. Sampling, sampling distributions, and parameter estimation
Lecture 3 Samplg, samplg dstrbutos, ad parameter estmato Samplg Defto Populato s defed as the collecto of all the possble observatos of terest. The collecto of observatos we take from the populato s called
More informationLecture Note to Rice Chapter 8
ECON 430 HG revsed Nov 06 Lecture Note to Rce Chapter 8 Radom matrces Let Y, =,,, m, =,,, be radom varables (r.v. s). The matrx Y Y Y Y Y Y Y Y Y Y = m m m s called a radom matrx ( wth a ot m-dmesoal dstrbuto,
More informationUnsupervised Learning and Other Neural Networks
CSE 53 Soft Computg NOT PART OF THE FINAL Usupervsed Learg ad Other Neural Networs Itroducto Mture Destes ad Idetfablty ML Estmates Applcato to Normal Mtures Other Neural Networs Itroducto Prevously, all
More informationPseudo-random Functions. PRG vs PRF
Pseudo-radom Fuctos Debdeep Muhopadhyay IIT Kharagpur PRG vs PRF We have see the costructo of PRG (pseudo-radom geerators) beg costructed from ay oe-way fuctos. Now we shall cosder a related cocept: Pseudo-radom
More informationArora Rao Vazirani Approximation for Expansion
Proof, belefs, ad algorthms through the les of sum-of-squares 1 Arora Rao Vazra Approxmato for Expaso I ths lecture, we cosder the problem of fdg a set wth smallest possble expaso 1 a gve graph. Earler
More informationMu Sequences/Series Solutions National Convention 2014
Mu Sequeces/Seres Solutos Natoal Coveto 04 C 6 E A 6C A 6 B B 7 A D 7 D C 7 A B 8 A B 8 A C 8 E 4 B 9 B 4 E 9 B 4 C 9 E C 0 A A 0 D B 0 C C Usg basc propertes of arthmetc sequeces, we fd a ad bm m We eed
More informationUNIT 2 SOLUTION OF ALGEBRAIC AND TRANSCENDENTAL EQUATIONS
Numercal Computg -I UNIT SOLUTION OF ALGEBRAIC AND TRANSCENDENTAL EQUATIONS Structure Page Nos..0 Itroducto 6. Objectves 7. Ital Approxmato to a Root 7. Bsecto Method 8.. Error Aalyss 9.4 Regula Fals Method
More informationLecture 2 - What are component and system reliability and how it can be improved?
Lecture 2 - What are compoet ad system relablty ad how t ca be mproved? Relablty s a measure of the qualty of the product over the log ru. The cocept of relablty s a exteded tme perod over whch the expected
More informationPPCP: The Proofs. 1 Notations and Assumptions. Maxim Likhachev Computer and Information Science University of Pennsylvania Philadelphia, PA 19104
PPCP: The Proofs Maxm Lkhachev Computer ad Iformato Scece Uversty of Pesylvaa Phladelpha, PA 19104 maxml@seas.upe.edu Athoy Stetz The Robotcs Isttute Carege Mello Uversty Pttsburgh, PA 15213 axs@rec.r.cmu.edu
More informationIncreasing Kolmogorov Complexity
Icreasg Kolmogorov Complexty Harry Buhrma Lace Fortow Ila Newma Nkola Vereshchag September 7, 2004 classfcato: Kolmogorov complexty, computatoal complexty 1 Itroducto How much do we have to chage a strg
More informationECE 729 Introduction to Channel Coding
chaelcodg.tex May 4, 2006 ECE 729 Itroducto to Chael Codg Cotets Fudametal Cocepts ad Techques. Chaels.....................2 Ecoders.....................2. Code Rates............... 2.3 Decoders....................
More informationComputational Geometry
Problem efto omputatoal eometry hapter 6 Pot Locato Preprocess a plaar map S. ve a query pot p, report the face of S cotag p. oal: O()-sze data structure that eables O(log ) query tme. pplcato: Whch state
More informationPr[X (p + t)n] e D KL(p+t p)n.
Cheroff Bouds Wolfgag Mulzer 1 The Geeral Boud Let P 1,..., m ) ad Q q 1,..., q m ) be two dstrbutos o m elemets,.e.,, q 0, for 1,..., m, ad m 1 m 1 q 1. The Kullback-Lebler dvergece or relatve etroy of
More informationLecture 07: Poles and Zeros
Lecture 07: Poles ad Zeros Defto of poles ad zeros The trasfer fucto provdes a bass for determg mportat system respose characterstcs wthout solvg the complete dfferetal equato. As defed, the trasfer fucto
More information2. Independence and Bernoulli Trials
. Ideedece ad Beroull Trals Ideedece: Evets ad B are deedet f B B. - It s easy to show that, B deedet mles, B;, B are all deedet ars. For examle, ad so that B or B B B B B φ,.e., ad B are deedet evets.,
More information10.1 Approximation Algorithms
290 0. Approxmato Algorthms Let us exame a problem, where we are gve A groud set U wth m elemets A collecto of subsets of the groud set = {,, } s.t. t s a cover of U: = U The am s to fd a subcover, = U,
More information5 Short Proofs of Simplified Stirling s Approximation
5 Short Proofs of Smplfed Strlg s Approxmato Ofr Gorodetsky, drtymaths.wordpress.com Jue, 20 0 Itroducto Strlg s approxmato s the followg (somewhat surprsg) approxmato of the factoral,, usg elemetary fuctos:
More information(b) By independence, the probability that the string 1011 is received correctly is
Soluto to Problem 1.31. (a) Let A be the evet that a 0 s trasmtted. Usg the total probablty theorem, the desred probablty s P(A)(1 ɛ ( 0)+ 1 P(A) ) (1 ɛ 1)=p(1 ɛ 0)+(1 p)(1 ɛ 1). (b) By depedece, the probablty
More informationExchangeable Sequences, Laws of Large Numbers, and the Mortgage Crisis.
Exchageable Sequeces, Laws of Large Numbers, ad the Mortgage Crss. Myug Joo Sog Advsor: Prof. Ja Madel May 2009 Itroducto The law of large umbers for..d. sequece gves covergece of sample meas to a costat,.e.,
More informationInvestigation of Partially Conditional RP Model with Response Error. Ed Stanek
Partally Codtoal Radom Permutato Model 7- vestgato of Partally Codtoal RP Model wth Respose Error TRODUCTO Ed Staek We explore the predctor that wll result a smple radom sample wth respose error whe a
More informationNon-uniform Turán-type problems
Joural of Combatoral Theory, Seres A 111 2005 106 110 wwwelsevercomlocatecta No-uform Turá-type problems DhruvMubay 1, Y Zhao 2 Departmet of Mathematcs, Statstcs, ad Computer Scece, Uversty of Illos at
More informationResearch Article A New Iterative Method for Common Fixed Points of a Finite Family of Nonexpansive Mappings
Hdaw Publshg Corporato Iteratoal Joural of Mathematcs ad Mathematcal Sceces Volume 009, Artcle ID 391839, 9 pages do:10.1155/009/391839 Research Artcle A New Iteratve Method for Commo Fxed Pots of a Fte
More informationDerivation of 3-Point Block Method Formula for Solving First Order Stiff Ordinary Differential Equations
Dervato of -Pot Block Method Formula for Solvg Frst Order Stff Ordary Dfferetal Equatos Kharul Hamd Kharul Auar, Kharl Iskadar Othma, Zara Bb Ibrahm Abstract Dervato of pot block method formula wth costat
More informationSTRONG CONSISTENCY OF LEAST SQUARES ESTIMATE IN MULTIPLE REGRESSION WHEN THE ERROR VARIANCE IS INFINITE
Statstca Sca 9(1999), 289-296 STRONG CONSISTENCY OF LEAST SQUARES ESTIMATE IN MULTIPLE REGRESSION WHEN THE ERROR VARIANCE IS INFINITE J Mgzhog ad Che Xru GuZhou Natoal College ad Graduate School, Chese
More informationTHE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA
THE ROYAL STATISTICAL SOCIETY 3 EXAMINATIONS SOLUTIONS GRADUATE DIPLOMA PAPER I STATISTICAL THEORY & METHODS The Socety provdes these solutos to assst caddates preparg for the examatos future years ad
More informationLog1 Contest Round 2 Theta Complex Numbers. 4 points each. 5 points each
01 Log1 Cotest Roud Theta Complex Numbers 1 Wrte a b Wrte a b form: 1 5 form: 1 5 4 pots each Wrte a b form: 65 4 4 Evaluate: 65 5 Determe f the followg statemet s always, sometmes, or ever true (you may
More informationLattices. Mathematical background
Lattces Mathematcal backgroud Lattces : -dmesoal Eucldea space. That s, { T x } x x = (,, ) :,. T T If x= ( x,, x), y = ( y,, y), the xy, = xy (er product of xad y) x = /2 xx, (Eucldea legth or orm of
More informationThird handout: On the Gini Index
Thrd hadout: O the dex Corrado, a tala statstca, proposed (, 9, 96) to measure absolute equalt va the mea dfferece whch s defed as ( / ) where refers to the total umber of dvduals socet. Assume that. The
More informationbest estimate (mean) for X uncertainty or error in the measurement (systematic, random or statistical) best
Error Aalyss Preamble Wheever a measuremet s made, the result followg from that measuremet s always subject to ucertaty The ucertaty ca be reduced by makg several measuremets of the same quatty or by mprovg
More informationIdea is to sample from a different distribution that picks points in important regions of the sample space. Want ( ) ( ) ( ) E f X = f x g x dx
Importace Samplg Used for a umber of purposes: Varace reducto Allows for dffcult dstrbutos to be sampled from. Sestvty aalyss Reusg samples to reduce computatoal burde. Idea s to sample from a dfferet
More informationMultiple Regression. More than 2 variables! Grade on Final. Multiple Regression 11/21/2012. Exam 2 Grades. Exam 2 Re-grades
STAT 101 Dr. Kar Lock Morga 11/20/12 Exam 2 Grades Multple Regresso SECTIONS 9.2, 10.1, 10.2 Multple explaatory varables (10.1) Parttog varablty R 2, ANOVA (9.2) Codtos resdual plot (10.2) Trasformatos
More informationNaïve Bayes MIT Course Notes Cynthia Rudin
Thaks to Şeyda Ertek Credt: Ng, Mtchell Naïve Bayes MIT 5.097 Course Notes Cytha Rud The Naïve Bayes algorthm comes from a geeratve model. There s a mportat dstcto betwee geeratve ad dscrmatve models.
More informationL5 Polynomial / Spline Curves
L5 Polyomal / Sple Curves Cotets Coc sectos Polyomal Curves Hermte Curves Bezer Curves B-Sples No-Uform Ratoal B-Sples (NURBS) Mapulato ad Represetato of Curves Types of Curve Equatos Implct: Descrbe a
More informationMS exam problems Fall 2012
MS exam problems Fall 01 (From: Rya Mart) 1. (Stat 401) Cosder the followg game wth a box that cotas te balls two red, three blue, ad fve gree. A player selects two balls from the box at radom, wthout
More informationLecture Notes Forecasting the process of estimating or predicting unknown situations
Lecture Notes. Ecoomc Forecastg. Forecastg the process of estmatg or predctg ukow stuatos Eample usuall ecoomsts predct future ecoomc varables Forecastg apples to a varet of data () tme seres data predctg
More informationBIOREPS Problem Set #11 The Evolution of DNA Strands
BIOREPS Problem Set #11 The Evoluto of DNA Strads 1 Backgroud I the md 2000s, evolutoary bologsts studyg DNA mutato rates brds ad prmates dscovered somethg surprsg. There were a large umber of mutatos
More informationJohns Hopkins University Department of Biostatistics Math Review for Introductory Courses
Johs Hopks Uverst Departmet of Bostatstcs Math Revew for Itroductor Courses Ratoale Bostatstcs courses wll rel o some fudametal mathematcal relatoshps, fuctos ad otato. The purpose of ths Math Revew s
More information( q Modal Analysis. Eigenvectors = Mode Shapes? Eigenproblem (cont) = x x 2 u 2. u 1. x 1 (4.55) vector and M and K are matrices.
4.3 - Modal Aalyss Physcal coordates are ot always the easest to work Egevectors provde a coveet trasformato to modal coordates Modal coordates are lear combato of physcal coordates Say we have physcal
More informationAssignment 7/MATH 247/Winter, 2010 Due: Friday, March 19. Powers of a square matrix
Assgmet 7/MATH 47/Wter, 00 Due: Frday, March 9 Powers o a square matrx Gve a square matrx A, ts powers A or large, or eve arbtrary, teger expoets ca be calculated by dagoalzg A -- that s possble (!) Namely,
More informationJohns Hopkins University Department of Biostatistics Math Review for Introductory Courses
Johs Hopks Uverst Departmet of Bostatstcs Math Revew for Itroductor Courses Ratoale Bostatstcs courses wll rel o some fudametal mathematcal relatoshps, fuctos ad otato. The purpose of ths Math Revew s
More informationLecture Notes to Rice Chapter 5
ECON 430 Revsed Sept. 06 Lecture Notes to Rce Chapter 5 By H. Goldste. Chapter 5 gves a troducto to probablstc approxmato methods, but s suffcet for the eeds of a adequate study of ecoometrcs. The commo
More informationFor combinatorial problems we might need to generate all permutations, combinations, or subsets of a set.
Addtoal Decrease ad Coquer Algorthms For combatoral problems we mght eed to geerate all permutatos, combatos, or subsets of a set. Geeratg Permutatos If we have a set f elemets: { a 1, a 2, a 3, a } the
More information1 Lyapunov Stability Theory
Lyapuov Stablty heory I ths secto we cosder proofs of stablty of equlbra of autoomous systems. hs s stadard theory for olear systems, ad oe of the most mportat tools the aalyss of olear systems. It may
More information