Stanford University CS254: Computational Complexity Notes 7 Luca Trevisan January 29, Notes for Lecture 7
|
|
- Loreen Little
- 6 years ago
- Views:
Transcription
1 Stanford Unversty CS54: Computatonal Complexty Notes 7 Luca Trevsan January 9, 014 Notes for Lecture 7 1 Approxmate Countng wt an N oracle We complete te proof of te followng result: Teorem 1 For every countng problem #A n #, tere s a probablstc algortm C tat on nput x, computes wt g probablty a value v suc tat (1 ɛ)#a(x) v (1 + ɛ)#a(x) (1) n tme polynomal n x and n 1 ɛ, usng an oracle for N. Gven wat we proved n te prevous lecture, t only remans to develop an approxmate comparson algortm for #CSAT, tat s, an algortm a comp suc tat for every crcut C: If #CSAT (C) k+1 ten a comp(c, k) = YES wt g probablty; If #CSAT (C) < k ten a comp(c, k) = NO wt g probablty. Te dea of te proof s to pck a random functon : {0, 1} n {0, 1} k, and ten consder te number of satsfyng assgnments for te crcut C (x) := C(x) ((x) = 0). If #CSAT (C) k+1 ten, on average over te coce of, C (x) as at least two satsfyng assgnments, but f #CSAT (C) k+1 ten, on average over te coce of, C (x) as less tan one satsfyng assgnments. Ceckng f C s satsfable s a test tat we would expect to dstngus te two cases. To make ts argument rgorous, we cannot pck te functon unformly at random among all functons, because ten would be an object requrng an exponental sze descrpton, and a descrpton of (n te form of an evaluaton algortm) as to be part of te crcut C. Instead we wll pck from a parwse ndependent dstrbuton of functons. To mprove te dstngusng probablty and smplfy te analyss, we wll work wt functons : {0, 1} n {0, 1} k 5, and we wll treat te case k 5 separately. 1.1 arwse ndependent as functons Defnton Let H be a dstrbuton over functons of te form : {0, 1} n {0, 1} m. We say tat H s a parwse ndependent dstrbuton of as functons f for every two dfferent nputs x, y {0, 1} n and for every two possble outputs s, t {0, 1} m we ave [(x) = s (y) = t] = 1 H m 1
2 Anoter way to look at te defnton s tat for every x y, wen we pck at random ten te random varables (x) and (y) are ndependent and unformly dstrbuted. In partcular, for every x y and for every s, t we ave [(x) = s (y) = t] = [(x) = s] A smple constructon of parwse ndependent as functons s as follows: pck a matrx A {0, 1} m n and a vector b {0, 1} m unformly at random, and ten defne te functon A,b (x) := Ax + b were te matrx product and vector addton operatons are performed over te feld F. (Tat s, tey are performed modulo.) To see tat te parwse ndependence property s satsfed, consder any two dstnct nputs x, y {0, 1} n and any two outputs s, t {0, 1} m. If we call a 1,..., a m te rows of A ten we ave [Ax + b = s Ay + b = t] = A,b m [a T x + b = s a T y + b = t ] =1 because te events (a T x + b = s a T y + b = t ) are all mutually ndependent. Te condton (a T x + b = s a T y + b = t ) can be equvalently rewrtten as and as a T x s = a T y t b = a T x s and ts probablty s a T (x y) = s t b = a T x s [a T (x y) = s t b = a T x s ] = [a T (x y) = s t ] [b = a T x s a T (x y) = s t ] = 1 1 = 1 4 Because x y s a non-zero vector, and so a T (x y) s a vector bt, and because b s a random bt ndependent of x, y. In concluson, we ave ( ) 1 m [Ax + b = s Ay + b = t] = A,b 4 as desred. We wll use te followng fact about parwse ndependent as functons.
3 Lemma 3 Let H be a dstrbuton of parwse ndependent as functons {0, 1} n {0, 1} m, and Let S {0, 1} n. Ten, for every t [ {a S : (a) = 0} S ] m t S t m. () H roof: We wll use Cebysev s Inequalty to bound te falure probablty. Let S = {a 1,..., a k }, and pck a random H. We defne random varables X 1,..., X k as X = { 1 f (a ) = 0 0 oterwse. (3) Clearly, {a S : (a) = 0} = X. We now calculate te expectatons. For eac, [X = 1] = 1 and E[X m ] = 1. Hence, m [ ] E X = S m. (4) Also we calculate te varance Var[X ] = E[X ] E[X ] E[X ] Because X 1,..., X k are parwse ndependent, [ ] Var X = = E[X ] = 1 m. Var[X ] S m. (5) Usng Cebysev s Inequalty, we get [ {a S : (a) = 0} S ] [ m t = X E[ Var[ X ] t = S t m ] X ] t] 1. Te algortm a-comp We defne te algortm a-comp as follows. nput: C, k f k 5 ten ceck exactly weter #CSAT (C) k. 3
4 f k 6 pck from a set of parwse ndependent as functons : {0, 1} n {0, 1} m, were m = k 5 answer YES ff tere are more ten 48 nputs x to C suc tat C(x) = 1 and (x) = 0. Notce tat te test at te last step can be done wt one access to an oracle to N and tat te overall algortm runs n probablstc polynomal tme gven an N oracle. We now analyze te correctness of te algortm. Let S {0, 1} n be te set of nputs x suc tat C(x) = 1. Tere are cases. If S k+1, let S S be an arbtrary subset of S of sze exactly k+1. S / m = 64 and we can use Lemma 3 to estmate te error probablty as: Ten [ {x S : (x) = 0} 48] H [ {x S : (x) = 0} 48] H [ S ] = {x S : (x) = 0} 16 H m 1 16 S m = 1 4 If S < k, ten S / m < 3, and te probablty of error can be estmated as [ {x S : (x) = 0} > 48] H [ ] {x S : (x) = 0} S 16 m H 1 16 S m 1 8 Terefore, te algortm wll gve te correct answer wt probablty at least 3/4, wc can ten be amplfed to, say, 1 1/4n (so tat all n nvocatons of a-comp are lkely to be correct) by repeatng te procedure O(log n) tmes and takng te majorty answer. Te Valant-Vazran Reducton We say tat an nstance C of crcut-sat s unquely satsfable f t as exactly one satsfyng assgnment. Valant and Vazran proved tat f tere s a polynomal tme randomzed algortm tat, gven a unquely satsfable crcut, fnds ts satsfyng assgnment, ten crcut-sat (and, ence, all oter problems n N) can be solved n randomzed polynomal tme. Te dea s related to te argument n te prevous secton. Gven a satsfable crcut C, we guess a number k suc tat k s approxmately te number of satsfyng assgnments 4
5 of C, we pck at random a parwse ndependent as functon : {0, 1} n {0, 1} k, and we construct te crcut C suc tat C (x) := C(x) ((x) = 0). Wt constant probablty, C as exactly one satsfyng assgnment, and ten te ypotetcal algortm tat solves unquely satsfable nstances wll fnd a satsfyng assgnment for C, and ence for C. It remans to prove tat f we ave a set S {0, 1} n, and we pck a parwse ndependent as functon : {0, 1} n {0, 1} k, were k log S, ten tere s a constant tat (x) = 0 for exactly one element of S. It wll not be possble to make ts argument work by usng Cebysev s nequalty, because wen te expected number of elements tat as to 0 s around 1, te standard devaton wll be too g. Instead, we use parwse ndependence to argue tat eac element of S as probablty Ω(1/ S ) of beng te unque element of S asng to 0; tese events are dsjont, and so ter probablty can be added up. Lemma 4 (Valant-Vazran) Let S {0, 1} n, let k be suc tat k S k+1, and let H be a famly of parwse ndependent as functons of te form : {0, 1} n {0, 1} k+. Ten f we pck at random from H, tere s probablty at least 1/8 tat tere s a unque element x S suc tat (x) = 0. recsely, r H [ {x S : (x) = 0} = 1] 1 8 (6) roof: For eac element x S, te probablty tat x s te unque element of S asng to 0 s [(x) = 0 ( y S {x}.(y) 0] Were = [(x) = 0] [(x) = 0 ( y S {x}.(y) = 0)] [(x) = 0] = 1 k+ and, usng a unon bound and parwse ndependence, [(x) = 0 ( y S {x}.(y) = 0)] y S {x} [(x) = (y) = 0] = S 1 k+4 1 k+3 Te probablty tat x s te unque element of S tat ases to 0 s tus [(x) = 0 ( y S {x}.(y) 0] 1 k+3 and te probablty tat a unque element of S ases to 0 s te sum of te above probabltes over all elements of S, and so t s at least S / k+3, wc s at least 1/8. We ave proved te followng result. Teorem 5 Suppose tat tere s a randomzed polynomal tme algortm A suc tat, gven a unquely satsfable crcut C, A fnds te satsfyng assgnment of C. Ten every problem n N s solvable n randomzed polynomal tme. 5
6 roof: It s enoug to sow tat, under te assumpton of te teorem, gven a (not necessarly unquely) satsfable crcut C, we can fnd a satsfyng assgnment for t n randomzed polynomal tme wt constant probablty. To do so, f n s te number of nputs of te gven crcut C, we try all k = 0,..., n, and for eac k we pck a parwse ndependent as functon k : {0, 1} n {0, 1} k+, and we run algortm A on te crcut C k (x) := C(x) ((x) = 0). For te coce of k suc tat te number of satsfyng assgnments of C s between k and k+1, we ave a constant probablty tat C k s unquely satsfable and tat A wll fnd a satsfyng assgnment. 3 Approxmate Samplng So far we ave consdered te followng queston: for an N-relaton R, gven an nput x, wat s te sze of te set R x = {y : (x, y) R}? A related queston s to be able to sample from te unform dstrbuton over R x. Wenever te relaton R s downward self reducble (a tecncal condton tat we won t defne formally), t s possble to prove tat tere s a probablstc algortm runnng n tme polynomal n x and 1/ɛ to approxmate wtn 1 + ɛ te value R x f and only f tere s a probablstc algortm runnng n tme polynomal n x and 1/ɛ tat samples a dstrbuton ɛ-close to te unform dstrbuton over R x. We sow ow te above result apples to 3SAT (te general result uses te same proof dea). For a formula φ, a varable x and a bt b, let us defne by φ x b te formula obtaned by substtutng te value b n place of x. 1 If φ s defned over varables x 1,..., x n, t s easy to see tat #φ = #φ x 0 + #φ x 1 Also, f S s te unform dstrbuton over satsfyng assgnments for φ, we note tat [x 1 = b] = #φ x b (x 1,...,x n) S #φ Suppose ten tat we ave an effcent samplng algortm tat gven φ and ɛ generates a dstrbuton ɛ-close to unform over te satsfyng assgnments of φ. Let us ten ran te samplng algortm wt approxmaton parameter ɛ/n and use t to sample about Õ(n /ɛ ) assgnments. By computng te fracton of suc assgnments avng x 1 = 0 and x 1 = 1, we get approxmate values p 0, p 1, suc tat p b (x1,...,x n) S[x 1 = b] ɛ/n. Let b be suc tat p b 1/, ten #φ x b /p b s a good approxmaton, to wtn a multplcatve factor (1 + ɛ/n) to #φ, and we can recurse to compute #φ x b to wtn a (1 + ɛ/n) n 1 factor. Conversely, suppose we ave an approxmate countng procedure. Ten we can approxmately compute p b = #φ x b #φ, generate a value b for x 1 wt probablty approxmately p b, and ten recurse to generate a random assgnment for #φ x b. Te same equvalence olds, clearly, for SAT and, among oter problems, for te problem of countng te number of perfect matcngs n a bpartte grap. It s known 1 Specfcally, φ x 1 s obtaned by removng eac occurrence of x from te clauses were t occurs, and removng all te clauses tat contan an occurrence of x; te formula φ x 0 s smlarly obtaned. 6
7 tat t s N-ard to perform approxmate countng for SAT and ts result, wt te above reducton, mples tat approxmate samplng s also ard for SAT. Te problem of approxmately samplng a perfect matcng as a probablstc polynomal soluton, and te reducton mples tat approxmately countng te number of perfect matcngs n a grap can also be done n probablstc polynomal tme. Te reducton and te results from last secton also mply tat 3SAT (and any oter N relaton) as an approxmate samplng algortm tat runs n probablstc polynomal tme wt an N oracle. Wt a careful use of te tecnques from last week t s ndeed possble to get an exact samplng algortm for 3SAT (and any oter N relaton) runnng n probablstc polynomal tme wt an N oracle. Ts s essentally best possble, because te approxmate samplng requres randomness by ts very defnton, and generatng satsfyng assgnments for a 3SAT formula requres at least an N oracle. 7
Lecture 4: Universal Hash Functions/Streaming Cont d
CSE 5: Desgn and Analyss of Algorthms I Sprng 06 Lecture 4: Unversal Hash Functons/Streamng Cont d Lecturer: Shayan Oves Gharan Aprl 6th Scrbe: Jacob Schreber Dsclamer: These notes have not been subjected
More information18.1 Introduction and Recap
CS787: Advanced Algorthms Scrbe: Pryananda Shenoy and Shjn Kong Lecturer: Shuch Chawla Topc: Streamng Algorthmscontnued) Date: 0/26/2007 We contnue talng about streamng algorthms n ths lecture, ncludng
More informationU.C. Berkeley CS294: Spectral Methods and Expanders Handout 8 Luca Trevisan February 17, 2016
U.C. Berkeley CS94: Spectral Methods and Expanders Handout 8 Luca Trevsan February 7, 06 Lecture 8: Spectral Algorthms Wrap-up In whch we talk about even more generalzatons of Cheeger s nequaltes, and
More informationStanford University CS359G: Graph Partitioning and Expanders Handout 4 Luca Trevisan January 13, 2011
Stanford Unversty CS359G: Graph Parttonng and Expanders Handout 4 Luca Trevsan January 3, 0 Lecture 4 In whch we prove the dffcult drecton of Cheeger s nequalty. As n the past lectures, consder an undrected
More informationMatrix Approximation via Sampling, Subspace Embedding. 1 Solving Linear Systems Using SVD
Matrx Approxmaton va Samplng, Subspace Embeddng Lecturer: Anup Rao Scrbe: Rashth Sharma, Peng Zhang 0/01/016 1 Solvng Lnear Systems Usng SVD Two applcatons of SVD have been covered so far. Today we loo
More informationNotes on Frequency Estimation in Data Streams
Notes on Frequency Estmaton n Data Streams In (one of) the data streamng model(s), the data s a sequence of arrvals a 1, a 2,..., a m of the form a j = (, v) where s the dentty of the tem and belongs to
More informationLecture 10: May 6, 2013
TTIC/CMSC 31150 Mathematcal Toolkt Sprng 013 Madhur Tulsan Lecture 10: May 6, 013 Scrbe: Wenje Luo In today s lecture, we manly talked about random walk on graphs and ntroduce the concept of graph expander,
More informationThe Finite Element Method: A Short Introduction
Te Fnte Element Metod: A Sort ntroducton Wat s FEM? Te Fnte Element Metod (FEM) ntroduced by engneers n late 50 s and 60 s s a numercal tecnque for solvng problems wc are descrbed by Ordnary Dfferental
More informationFinding Primitive Roots Pseudo-Deterministically
Electronc Colloquum on Computatonal Complexty, Report No 207 (205) Fndng Prmtve Roots Pseudo-Determnstcally Ofer Grossman December 22, 205 Abstract Pseudo-determnstc algorthms are randomzed search algorthms
More informationLecture Space-Bounded Derandomization
Notes on Complexty Theory Last updated: October, 2008 Jonathan Katz Lecture Space-Bounded Derandomzaton 1 Space-Bounded Derandomzaton We now dscuss derandomzaton of space-bounded algorthms. Here non-trval
More informationFinding Dense Subgraphs in G(n, 1/2)
Fndng Dense Subgraphs n Gn, 1/ Atsh Das Sarma 1, Amt Deshpande, and Rav Kannan 1 Georga Insttute of Technology,atsh@cc.gatech.edu Mcrosoft Research-Bangalore,amtdesh,annan@mcrosoft.com Abstract. Fndng
More informationU.C. Berkeley CS278: Computational Complexity Professor Luca Trevisan 2/21/2008. Notes for Lecture 8
U.C. Berkeley CS278: Computatonal Complexty Handout N8 Professor Luca Trevsan 2/21/2008 Notes for Lecture 8 1 Undrected Connectvty In the undrected s t connectvty problem (abbrevated ST-UCONN) we are gven
More informationOn Pfaff s solution of the Pfaff problem
Zur Pfaff scen Lösung des Pfaff scen Probles Mat. Ann. 7 (880) 53-530. On Pfaff s soluton of te Pfaff proble By A. MAYER n Lepzg Translated by D. H. Delpenc Te way tat Pfaff adopted for te ntegraton of
More informationprinceton univ. F 13 cos 521: Advanced Algorithm Design Lecture 3: Large deviations bounds and applications Lecturer: Sanjeev Arora
prnceton unv. F 13 cos 521: Advanced Algorthm Desgn Lecture 3: Large devatons bounds and applcatons Lecturer: Sanjeev Arora Scrbe: Today s topc s devaton bounds: what s the probablty that a random varable
More informationU.C. Berkeley CS294: Beyond Worst-Case Analysis Handout 6 Luca Trevisan September 12, 2017
U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 6 Luca Trevsan September, 07 Scrbed by Theo McKenze Lecture 6 In whch we study the spectrum of random graphs. Overvew When attemptng to fnd n polynomal
More information6.842 Randomness and Computation February 18, Lecture 4
6.842 Randomness and Computaton February 18, 2014 Lecture 4 Lecturer: Rontt Rubnfeld Scrbe: Amartya Shankha Bswas Topcs 2-Pont Samplng Interactve Proofs Publc cons vs Prvate cons 1 Two Pont Samplng 1.1
More information11 Tail Inequalities Markov s Inequality. Lecture 11: Tail Inequalities [Fa 13]
Algorthms Lecture 11: Tal Inequaltes [Fa 13] If you hold a cat by the tal you learn thngs you cannot learn any other way. Mark Twan 11 Tal Inequaltes The smple recursve structure of skp lsts made t relatvely
More informationLectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix
Lectures - Week 4 Matrx norms, Condtonng, Vector Spaces, Lnear Independence, Spannng sets and Bass, Null space and Range of a Matrx Matrx Norms Now we turn to assocatng a number to each matrx. We could
More informationExpected Value and Variance
MATH 38 Expected Value and Varance Dr. Neal, WKU We now shall dscuss how to fnd the average and standard devaton of a random varable X. Expected Value Defnton. The expected value (or average value, or
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 12 10/21/2013. Martingale Concentration Inequalities and Applications
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.65/15.070J Fall 013 Lecture 1 10/1/013 Martngale Concentraton Inequaltes and Applcatons Content. 1. Exponental concentraton for martngales wth bounded ncrements.
More informationE Tail Inequalities. E.1 Markov s Inequality. Non-Lecture E: Tail Inequalities
Algorthms Non-Lecture E: Tal Inequaltes If you hold a cat by the tal you learn thngs you cannot learn any other way. Mar Twan E Tal Inequaltes The smple recursve structure of sp lsts made t relatvely easy
More informationThe Multiple Classical Linear Regression Model (CLRM): Specification and Assumptions. 1. Introduction
ECONOMICS 5* -- NOTE (Summary) ECON 5* -- NOTE The Multple Classcal Lnear Regresson Model (CLRM): Specfcaton and Assumptons. Introducton CLRM stands for the Classcal Lnear Regresson Model. The CLRM s also
More informationAdaptive Kernel Estimation of the Conditional Quantiles
Internatonal Journal of Statstcs and Probablty; Vol. 5, No. ; 206 ISSN 927-7032 E-ISSN 927-7040 Publsed by Canadan Center of Scence and Educaton Adaptve Kernel Estmaton of te Condtonal Quantles Rad B.
More informationTR/95 February Splines G. H. BEHFOROOZ* & N. PAPAMICHAEL
TR/9 February 980 End Condtons for Interpolatory Quntc Splnes by G. H. BEHFOROOZ* & N. PAPAMICHAEL *Present address: Dept of Matematcs Unversty of Tabrz Tabrz Iran. W9609 A B S T R A C T Accurate end condtons
More informationCOMP4630: λ-calculus
COMP4630: λ-calculus 4. Standardsaton Mcael Norrs Mcael.Norrs@ncta.com.au Canberra Researc Lab., NICTA Semester 2, 2015 Last Tme Confluence Te property tat dvergent evaluatons can rejon one anoter Proof
More informationMultivariate Ratio Estimator of the Population Total under Stratified Random Sampling
Open Journal of Statstcs, 0,, 300-304 ttp://dx.do.org/0.436/ojs.0.3036 Publsed Onlne July 0 (ttp://www.scrp.org/journal/ojs) Multvarate Rato Estmator of te Populaton Total under Stratfed Random Samplng
More informationProblem Set 4: Sketch of Solutions
Problem Set 4: Sketc of Solutons Informaton Economcs (Ec 55) George Georgads Due n class or by e-mal to quel@bu.edu at :30, Monday, December 8 Problem. Screenng A monopolst can produce a good n dfferent
More informationj) = 1 (note sigma notation) ii. Continuous random variable (e.g. Normal distribution) 1. density function: f ( x) 0 and f ( x) dx = 1
Random varables Measure of central tendences and varablty (means and varances) Jont densty functons and ndependence Measures of assocaton (covarance and correlaton) Interestng result Condtonal dstrbutons
More informationEigenvalues of Random Graphs
Spectral Graph Theory Lecture 2 Egenvalues of Random Graphs Danel A. Spelman November 4, 202 2. Introducton In ths lecture, we consder a random graph on n vertces n whch each edge s chosen to be n the
More informationStanford University CS254: Computational Complexity Handout 8 Luca Trevisan 4/21/2010
Stanford University CS254: Computational Complexity Handout 8 Luca Trevisan 4/2/200 Counting Problems Today we describe counting problems and the class #P that they define, and we show that every counting
More informationAPPENDIX A Some Linear Algebra
APPENDIX A Some Lnear Algebra The collecton of m, n matrces A.1 Matrces a 1,1,..., a 1,n A = a m,1,..., a m,n wth real elements a,j s denoted by R m,n. If n = 1 then A s called a column vector. Smlarly,
More informationMaximizing the number of nonnegative subsets
Maxmzng the number of nonnegatve subsets Noga Alon Hao Huang December 1, 213 Abstract Gven a set of n real numbers, f the sum of elements of every subset of sze larger than k s negatve, what s the maxmum
More informationPart (a) (Number of collisions) Recall we showed that if we throw m balls in n bins, the average number of. Use Chebyshev s inequality to show that:
Problem 1: Practce wth Chebyshev and Chernoff bounds) When usng concentraton bounds to analyze randomzed algorthms, one often has to approach the problem n dfferent ways dependng on the specfc bound beng
More informationLecture 3: Probability Distributions
Lecture 3: Probablty Dstrbutons Random Varables Let us begn by defnng a sample space as a set of outcomes from an experment. We denote ths by S. A random varable s a functon whch maps outcomes nto the
More informationMath 426: Probability MWF 1pm, Gasson 310 Homework 4 Selected Solutions
Exercses from Ross, 3, : Math 26: Probablty MWF pm, Gasson 30 Homework Selected Solutons 3, p. 05 Problems 76, 86 3, p. 06 Theoretcal exercses 3, 6, p. 63 Problems 5, 0, 20, p. 69 Theoretcal exercses 2,
More informationNP-Completeness : Proofs
NP-Completeness : Proofs Proof Methods A method to show a decson problem Π NP-complete s as follows. (1) Show Π NP. (2) Choose an NP-complete problem Π. (3) Show Π Π. A method to show an optmzaton problem
More informationU.C. Berkeley CS294: Beyond Worst-Case Analysis Luca Trevisan September 5, 2017
U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 4s Luca Trevsan September 5, 07 Summary of Lecture 4 In whch we ntroduce semdefnte programmng and apply t to Max Cut. Semdefnte Programmng Recall that
More informationFirst Year Examination Department of Statistics, University of Florida
Frst Year Examnaton Department of Statstcs, Unversty of Florda May 7, 010, 8:00 am - 1:00 noon Instructons: 1. You have four hours to answer questons n ths examnaton.. You must show your work to receve
More informationConvergence of random processes
DS-GA 12 Lecture notes 6 Fall 216 Convergence of random processes 1 Introducton In these notes we study convergence of dscrete random processes. Ths allows to characterze phenomena such as the law of large
More informationIntroduction to Algorithms
Introducton to Algorthms 6.046J/8.40J Lecture 7 Prof. Potr Indyk Data Structures Role of data structures: Encapsulate data Support certan operatons (e.g., INSERT, DELETE, SEARCH) Our focus: effcency of
More information3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X
Statstcs 1: Probablty Theory II 37 3 EPECTATION OF SEVERAL RANDOM VARIABLES As n Probablty Theory I, the nterest n most stuatons les not on the actual dstrbuton of a random vector, but rather on a number
More informationEstimation: Part 2. Chapter GREG estimation
Chapter 9 Estmaton: Part 2 9. GREG estmaton In Chapter 8, we have seen that the regresson estmator s an effcent estmator when there s a lnear relatonshp between y and x. In ths chapter, we generalzed the
More informationarxiv: v1 [math.co] 1 Mar 2014
Unon-ntersectng set systems Gyula O.H. Katona and Dánel T. Nagy March 4, 014 arxv:1403.0088v1 [math.co] 1 Mar 014 Abstract Three ntersecton theorems are proved. Frst, we determne the sze of the largest
More informationCase A. P k = Ni ( 2L i k 1 ) + (# big cells) 10d 2 P k.
THE CELLULAR METHOD In ths lecture, we ntroduce the cellular method as an approach to ncdence geometry theorems lke the Szemeréd-Trotter theorem. The method was ntroduced n the paper Combnatoral complexty
More informationECE 534: Elements of Information Theory. Solutions to Midterm Exam (Spring 2006)
ECE 534: Elements of Informaton Theory Solutons to Mdterm Eam (Sprng 6) Problem [ pts.] A dscrete memoryless source has an alphabet of three letters,, =,, 3, wth probabltes.4,.4, and., respectvely. (a)
More informationPubH 7405: REGRESSION ANALYSIS. SLR: INFERENCES, Part II
PubH 7405: REGRESSION ANALSIS SLR: INFERENCES, Part II We cover te topc of nference n two sessons; te frst sesson focused on nferences concernng te slope and te ntercept; ts s a contnuaton on estmatng
More informationLecture Randomized Load Balancing strategies and their analysis. Probability concepts include, counting, the union bound, and Chernoff bounds.
U.C. Berkeley CS273: Parallel and Dstrbuted Theory Lecture 1 Professor Satsh Rao August 26, 2010 Lecturer: Satsh Rao Last revsed September 2, 2010 Lecture 1 1 Course Outlne We wll cover a samplng of the
More informationn α j x j = 0 j=1 has a nontrivial solution. Here A is the n k matrix whose jth column is the vector for all t j=0
MODULE 2 Topcs: Lnear ndependence, bass and dmenson We have seen that f n a set of vectors one vector s a lnear combnaton of the remanng vectors n the set then the span of the set s unchanged f that vector
More informationLecture 4: Constant Time SVD Approximation
Spectral Algorthms and Representatons eb. 17, Mar. 3 and 8, 005 Lecture 4: Constant Tme SVD Approxmaton Lecturer: Santosh Vempala Scrbe: Jangzhuo Chen Ths topc conssts of three lectures 0/17, 03/03, 03/08),
More informationErrors for Linear Systems
Errors for Lnear Systems When we solve a lnear system Ax b we often do not know A and b exactly, but have only approxmatons  and ˆb avalable. Then the best thng we can do s to solve ˆx ˆb exactly whch
More informationCompetitive Experimentation and Private Information
Compettve Expermentaton an Prvate Informaton Guseppe Moscarn an Francesco Squntan Omtte Analyss not Submtte for Publcaton Dervatons for te Gamma-Exponental Moel Dervaton of expecte azar rates. By Bayes
More informationStanford University Graph Partitioning and Expanders Handout 3 Luca Trevisan May 8, 2013
Stanford Unversty Graph Parttonng and Expanders Handout 3 Luca Trevsan May 8, 03 Lecture 3 In whch we analyze the power method to approxmate egenvalues and egenvectors, and we descrbe some more algorthmc
More informationLecture 5 September 17, 2015
CS 229r: Algorthms for Bg Data Fall 205 Prof. Jelan Nelson Lecture 5 September 7, 205 Scrbe: Yakr Reshef Recap and overvew Last tme we dscussed the problem of norm estmaton for p-norms wth p > 2. We had
More informationLecture 14 (03/27/18). Channels. Decoding. Preview of the Capacity Theorem.
Lecture 14 (03/27/18). Channels. Decodng. Prevew of the Capacty Theorem. A. Barg The concept of a communcaton channel n nformaton theory s an abstracton for transmttng dgtal (and analog) nformaton from
More informationMin Cut, Fast Cut, Polynomial Identities
Randomzed Algorthms, Summer 016 Mn Cut, Fast Cut, Polynomal Identtes Instructor: Thomas Kesselhem and Kurt Mehlhorn 1 Mn Cuts n Graphs Lecture (5 pages) Throughout ths secton, G = (V, E) s a mult-graph.
More informationEconomics 130. Lecture 4 Simple Linear Regression Continued
Economcs 130 Lecture 4 Contnued Readngs for Week 4 Text, Chapter and 3. We contnue wth addressng our second ssue + add n how we evaluate these relatonshps: Where do we get data to do ths analyss? How do
More informationHomework Assignment 3 Due in class, Thursday October 15
Homework Assgnment 3 Due n class, Thursday October 15 SDS 383C Statstcal Modelng I 1 Rdge regresson and Lasso 1. Get the Prostrate cancer data from http://statweb.stanford.edu/~tbs/elemstatlearn/ datasets/prostate.data.
More informationLecture 5 Decoding Binary BCH Codes
Lecture 5 Decodng Bnary BCH Codes In ths class, we wll ntroduce dfferent methods for decodng BCH codes 51 Decodng the [15, 7, 5] 2 -BCH Code Consder the [15, 7, 5] 2 -code C we ntroduced n the last lecture
More informationEcon107 Applied Econometrics Topic 3: Classical Model (Studenmund, Chapter 4)
I. Classcal Assumptons Econ7 Appled Econometrcs Topc 3: Classcal Model (Studenmund, Chapter 4) We have defned OLS and studed some algebrac propertes of OLS. In ths topc we wll study statstcal propertes
More informationMATH 5707 HOMEWORK 4 SOLUTIONS 2. 2 i 2p i E(X i ) + E(Xi 2 ) ä i=1. i=1
MATH 5707 HOMEWORK 4 SOLUTIONS CİHAN BAHRAN 1. Let v 1,..., v n R m, all lengths v are not larger than 1. Let p 1,..., p n [0, 1] be arbtrary and set w = p 1 v 1 + + p n v n. Then there exst ε 1,..., ε
More informationRandomness and Computation
Randomness and Computaton or, Randomzed Algorthms Mary Cryan School of Informatcs Unversty of Ednburgh RC 208/9) Lecture 0 slde Balls n Bns m balls, n bns, and balls thrown unformly at random nto bns usually
More informationSingular Value Decomposition: Theory and Applications
Sngular Value Decomposton: Theory and Applcatons Danel Khashab Sprng 2015 Last Update: March 2, 2015 1 Introducton A = UDV where columns of U and V are orthonormal and matrx D s dagonal wth postve real
More informationHardness of Learning Halfspaces with Noise
Hardness of Learnng Halfspaces wth Nose Venkatesan Guruswam Prasad Raghavendra Department of Computer Scence and Engneerng Unversty of Washngton Seattle, WA 98195 Abstract Learnng an unknown halfspace
More informationLECTURE 5: FIBRATIONS AND HOMOTOPY FIBERS
LECTURE 5: FIBRATIONS AND HOMOTOPY FIBERS In ts lecture we wll ntroduce two mortant classes of mas of saces, namely te Hurewcz fbratons and te more general Serre fbratons, wc are bot obtaned by mosng certan
More informationExercises of Chapter 2
Exercses of Chapter Chuang-Cheh Ln Department of Computer Scence and Informaton Engneerng, Natonal Chung Cheng Unversty, Mng-Hsung, Chay 61, Tawan. Exercse.6. Suppose that we ndependently roll two standard
More informationThe Order Relation and Trace Inequalities for. Hermitian Operators
Internatonal Mathematcal Forum, Vol 3, 08, no, 507-57 HIKARI Ltd, wwwm-hkarcom https://doorg/0988/mf088055 The Order Relaton and Trace Inequaltes for Hermtan Operators Y Huang School of Informaton Scence
More informationNUMERICAL DIFFERENTIATION
NUMERICAL DIFFERENTIATION 1 Introducton Dfferentaton s a method to compute the rate at whch a dependent output y changes wth respect to the change n the ndependent nput x. Ths rate of change s called the
More informationA Discrete Approach to Continuous Second-Order Boundary Value Problems via Monotone Iterative Techniques
Internatonal Journal of Dfference Equatons ISSN 0973-6069, Volume 12, Number 1, pp. 145 160 2017) ttp://campus.mst.edu/jde A Dscrete Approac to Contnuous Second-Order Boundary Value Problems va Monotone
More informationMath 217 Fall 2013 Homework 2 Solutions
Math 17 Fall 013 Homework Solutons Due Thursday Sept. 6, 013 5pm Ths homework conssts of 6 problems of 5 ponts each. The total s 30. You need to fully justfy your answer prove that your functon ndeed has
More informationNote on quantum counting classes
Note on quantum countng classes Yaoyun Sh Shengyu Zhang Abstract We defne countng classes #BPP and #BQP as natural extensons of the classcal well-studed one #P to the randomzed and quantum cases. It s
More informationProblem Set 9 Solutions
Desgn and Analyss of Algorthms May 4, 2015 Massachusetts Insttute of Technology 6.046J/18.410J Profs. Erk Demane, Srn Devadas, and Nancy Lynch Problem Set 9 Solutons Problem Set 9 Solutons Ths problem
More informationFINITELY-GENERATED MODULES OVER A PRINCIPAL IDEAL DOMAIN
FINITELY-GENERTED MODULES OVER PRINCIPL IDEL DOMIN EMMNUEL KOWLSKI Throughout ths note, s a prncpal deal doman. We recall the classfcaton theorem: Theorem 1. Let M be a fntely-generated -module. (1) There
More informationTAIL BOUNDS FOR SUMS OF GEOMETRIC AND EXPONENTIAL VARIABLES
TAIL BOUNDS FOR SUMS OF GEOMETRIC AND EXPONENTIAL VARIABLES SVANTE JANSON Abstract. We gve explct bounds for the tal probabltes for sums of ndependent geometrc or exponental varables, possbly wth dfferent
More information10-701/ Machine Learning, Fall 2005 Homework 3
10-701/15-781 Machne Learnng, Fall 2005 Homework 3 Out: 10/20/05 Due: begnnng of the class 11/01/05 Instructons Contact questons-10701@autonlaborg for queston Problem 1 Regresson and Cross-valdaton [40
More information5 The Laplace Equation in a convex polygon
5 Te Laplace Equaton n a convex polygon Te most mportant ellptc PDEs are te Laplace, te modfed Helmoltz and te Helmoltz equatons. Te Laplace equaton s u xx + u yy =. (5.) Te real and magnary parts of an
More informationAppendix B. Criterion of Riemann-Stieltjes Integrability
Appendx B. Crteron of Remann-Steltes Integrablty Ths note s complementary to [R, Ch. 6] and [T, Sec. 3.5]. The man result of ths note s Theorem B.3, whch provdes the necessary and suffcent condtons for
More informationOn a nonlinear compactness lemma in L p (0, T ; B).
On a nonlnear compactness lemma n L p (, T ; B). Emmanuel Matre Laboratore de Matématques et Applcatons Unversté de Haute-Alsace 4, rue des Frères Lumère 6893 Mulouse E.Matre@ua.fr 3t February 22 Abstract
More informationCalculation of time complexity (3%)
Problem 1. (30%) Calculaton of tme complexty (3%) Gven n ctes, usng exhaust search to see every result takes O(n!). Calculaton of tme needed to solve the problem (2%) 40 ctes:40! dfferent tours 40 add
More informationLinear Regression Analysis: Terminology and Notation
ECON 35* -- Secton : Basc Concepts of Regresson Analyss (Page ) Lnear Regresson Analyss: Termnology and Notaton Consder the generc verson of the smple (two-varable) lnear regresson model. It s represented
More information8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS
SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 493 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces you have studed thus far n the text are real vector spaces because the scalars
More informationFeature Selection: Part 1
CSE 546: Machne Learnng Lecture 5 Feature Selecton: Part 1 Instructor: Sham Kakade 1 Regresson n the hgh dmensonal settng How do we learn when the number of features d s greater than the sample sze n?
More informationIntroduction to Algorithms
Introducton to Algorthms 6.046J/18.401J Lecture 7 Prof. Potr Indyk Data Structures Role of data structures: Encapsulate data Support certan operatons (e.g., INSERT, DELETE, SEARCH) What data structures
More informationLecture 3 January 31, 2017
CS 224: Advanced Algorthms Sprng 207 Prof. Jelan Nelson Lecture 3 January 3, 207 Scrbe: Saketh Rama Overvew In the last lecture we covered Y-fast tres and Fuson Trees. In ths lecture we start our dscusson
More informationTornado and Luby Transform Codes. Ashish Khisti Presentation October 22, 2003
Tornado and Luby Transform Codes Ashsh Khst 6.454 Presentaton October 22, 2003 Background: Erasure Channel Elas[956] studed the Erasure Channel β x x β β x 2 m x 2 k? Capacty of Noseless Erasure Channel
More informationANSWERS. Problem 1. and the moment generating function (mgf) by. defined for any real t. Use this to show that E( U) var( U)
Econ 413 Exam 13 H ANSWERS Settet er nndelt 9 deloppgaver, A,B,C, som alle anbefales å telle lkt for å gøre det ltt lettere å stå. Svar er gtt . Unfortunately, there s a prntng error n the hnt of
More informationCollege of Computer & Information Science Fall 2009 Northeastern University 20 October 2009
College of Computer & Informaton Scence Fall 2009 Northeastern Unversty 20 October 2009 CS7880: Algorthmc Power Tools Scrbe: Jan Wen and Laura Poplawsk Lecture Outlne: Prmal-dual schema Network Desgn:
More informationLow correlation tensor decomposition via entropy maximization
CS369H: Herarches of Integer Programmng Relaxatons Sprng 2016-2017 Low correlaton tensor decomposton va entropy maxmzaton Lecture and notes by Sam Hopkns Scrbes: James Hong Overvew CS369H). These notes
More informationThe Second Anti-Mathima on Game Theory
The Second Ant-Mathma on Game Theory Ath. Kehagas December 1 2006 1 Introducton In ths note we wll examne the noton of game equlbrum for three types of games 1. 2-player 2-acton zero-sum games 2. 2-player
More informationA note on almost sure behavior of randomly weighted sums of φ-mixing random variables with φ-mixing weights
ACTA ET COMMENTATIONES UNIVERSITATIS TARTUENSIS DE MATHEMATICA Volume 7, Number 2, December 203 Avalable onlne at http://acutm.math.ut.ee A note on almost sure behavor of randomly weghted sums of φ-mxng
More informationfind (x): given element x, return the canonical element of the set containing x;
COS 43 Sprng, 009 Dsjont Set Unon Problem: Mantan a collecton of dsjont sets. Two operatons: fnd the set contanng a gven element; unte two sets nto one (destructvely). Approach: Canoncal element method:
More informationA new construction of 3-separable matrices via an improved decoding of Macula s construction
Dscrete Optmzaton 5 008 700 704 Contents lsts avalable at ScenceDrect Dscrete Optmzaton journal homepage: wwwelsevercom/locate/dsopt A new constructon of 3-separable matrces va an mproved decodng of Macula
More informationLecture 3. Ax x i a i. i i
18.409 The Behavor of Algorthms n Practce 2/14/2 Lecturer: Dan Spelman Lecture 3 Scrbe: Arvnd Sankar 1 Largest sngular value In order to bound the condton number, we need an upper bound on the largest
More informationChapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems
Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons
More informationEcon Statistical Properties of the OLS estimator. Sanjaya DeSilva
Econ 39 - Statstcal Propertes of the OLS estmator Sanjaya DeSlva September, 008 1 Overvew Recall that the true regresson model s Y = β 0 + β 1 X + u (1) Applyng the OLS method to a sample of data, we estmate
More informationTransfer Functions. Convenient representation of a linear, dynamic model. A transfer function (TF) relates one input and one output: ( ) system
Transfer Functons Convenent representaton of a lnear, dynamc model. A transfer functon (TF) relates one nput and one output: x t X s y t system Y s The followng termnology s used: x y nput output forcng
More informationLOW BIAS INTEGRATED PATH ESTIMATORS. James M. Calvin
Proceedngs of the 007 Wnter Smulaton Conference S G Henderson, B Bller, M-H Hseh, J Shortle, J D Tew, and R R Barton, eds LOW BIAS INTEGRATED PATH ESTIMATORS James M Calvn Department of Computer Scence
More informationComputing Correlated Equilibria in Multi-Player Games
Computng Correlated Equlbra n Mult-Player Games Chrstos H. Papadmtrou Presented by Zhanxang Huang December 7th, 2005 1 The Author Dr. Chrstos H. Papadmtrou CS professor at UC Berkley (taught at Harvard,
More informationSolution for singularly perturbed problems via cubic spline in tension
ISSN 76-769 England UK Journal of Informaton and Computng Scence Vol. No. 06 pp.6-69 Soluton for sngularly perturbed problems va cubc splne n tenson K. Aruna A. S. V. Rav Kant Flud Dynamcs Dvson Scool
More informationCS 2750 Machine Learning. Lecture 5. Density estimation. CS 2750 Machine Learning. Announcements
CS 750 Machne Learnng Lecture 5 Densty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square CS 750 Machne Learnng Announcements Homework Due on Wednesday before the class Reports: hand n before
More informationTHE CHINESE REMAINDER THEOREM. We should thank the Chinese for their wonderful remainder theorem. Glenn Stevens
THE CHINESE REMAINDER THEOREM KEITH CONRAD We should thank the Chnese for ther wonderful remander theorem. Glenn Stevens 1. Introducton The Chnese remander theorem says we can unquely solve any par of
More information