Introduction to Algorithms
|
|
- Cuthbert Warner
- 6 years ago
- Views:
Transcription
1 Introducton to Algorthms 6.046J/18.401J Lecture 7 Prof. Potr Indyk
2 Data Structures Role of data structures: Encapsulate data Support certan operatons (e.g., INSERT, DELETE, SEARCH) What data structures do we know already? Yes, heap: INSERT(x) DELETE-MIN Introducton to Algorthms September 29, 2004 L7.2
3 Dctonary problem Dctonary T holdng n records: x record key[x] Other felds contanng satellte data Operatons on T: INSERT(T, x) DELETE(T, x) SEARCH(T, k) How should the data structure T be organzed? Introducton to Algorthms September 29, 2004 L7.3
4 Assumptons Assumptons: The set of keys s K U = {0, 1,, u 1} Keys are dstnct What can we do? Introducton to Algorthms September 29, 2004 L7.4
5 Drect access table Create a table T[0 u-1]: T[k] = x f k K and key[x] = k, NIL otherwse. Beneft: Each operaton takes constant tme Drawbacks: The range of keys can be large: 64-bt numbers (whch represent 18,446,744,073,709,551,616 dfferent keys), character strngs (even larger!) Introducton to Algorthms September 29, 2004 L7.5
6 Hash functons Soluton: Use a hash functon h to map the unverse U of all keys nto T {0, 1,, m 1}: K k 1 k 5 k 4 k 2 k 3 U When a record to be nserted maps to an already As occuped each key slot s n nserted, T, a collson h maps occurs. t to a slot of T. Introducton to Algorthms September 29, 2004 L7.6 0 h(k 1 ) h(k 4 ) h(k 2 ) = h(k 5 ) h(k 3 ) m 1
7 Collsons resoluton by channg Records n the same slot are lnked nto a lst. T h(49) = h(86) = h(52) = Introducton to Algorthms September 29, 2004 L7.7
8 Hash functons Desgnng good functons s qute nontrval For now, we assume they exst. Namely, we assume smple unform hashng: Each key k K of keys s equally lkely to be hashed to any slot of table T, ndependent of where other keys are hashed Introducton to Algorthms September 29, 2004 L7.8
9 Analyss of channg Let n be the number of keys n the table, and let m be the number of slots. Defne the load factor of T to be α = n/m = average number of keys per slot. Introducton to Algorthms September 29, 2004 L7.9
10 Search cost Expected tme to search for a record wth a gven key = Θ(1 + α). apply hash functon and access slot search the lst Expected search tme = Θ(1) f α = O(1), or equvalently, f n = O(m). Introducton to Algorthms September 29, 2004 L7.10
11 Other operatons Inserton tme? Constant: hash and add to the lst Deleton tme? Recall that we defned DELETE(T, x) Also constant, f x has a ponter to the collson lst and lst doubly lnked Otherwse, do SEARCH frst Introducton to Algorthms September 29, 2004 L7.11
12 Delete key[x] T Introducton to Algorthms September 29, 2004 L7.12
13 Dealng wth wshful thnkng The assumpton of smple unform hashng s hard to guarantee, but several common technques tend to work well n practce as long as ther defcences can be avoded. Desrata: A good hash functon should dstrbute the keys unformly nto the slots of the table. Regularty n the key dstrbuton (e.g., arthmetc progresson) should not affect ths unformty. Introducton to Algorthms September 29, 2004 L7.13
14 Hashng n practce Leavng the realm of Provable Introducton to Algorthms September 29, 2004 L7.14
15 Dvson method Defne h(k) = k mod m. Defcency: Don t pck an m that has a small dvsor d. A preponderance of keys that are congruent modulo d can adversely affect unformty. Extreme defcency: If m = 2 r, then the hash doesn t even depend on all the bts of k: If k = and r = 6, then h(k) = h(k) Introducton to Algorthms September 29, 2004 L7.15
16 Dvson method (contnued) Pck m to be a prme. h(k) = k mod m. Annoyance: Sometmes, makng the table sze a prme s nconvenent. But, ths method s popular, although the next method we ll see s usually superor. Introducton to Algorthms September 29, 2004 L7.16
17 Multplcaton method Assume that all keys are ntegers, m = 2 r, and our computer has w-bt words. Defne h(k) = (A k mod 2 w ) rsh (w r), where rsh s the bt-wse rght-shft operator and A s an odd nteger n the range 2 w 1 < A < 2 w. Don t pck A too close to 2 w. Multplcaton modulo 2 w s fast. The rsh operator s fast. Introducton to Algorthms September 29, 2004 L7.17
18 Multplcaton method example h(k) = (A k mod 2 w ) rsh (w r) Suppose that m = 8 = 2 3 and that our computer has w = 7-bt words: 3A = A = k h(k) A. Modular wheel A Introducton to Algorthms September 29, 2004 L7.18
19 Back to the realm of Provable Introducton to Algorthms September 29, 2004 L7.19
20 A weakness of hashng as we saw t Problem: For any hash functon h, a set of keys exsts that can cause the average access tme of a hash table to skyrocket. An adversary can pck all keys from h -1 () ={k U : h(k) = } for a slot. There s a slot for whch h -1 () u/m Introducton to Algorthms September 29, 2004 L7.20
21 Soluton Randomze! Choose the hash functon at random from some famly of functon, and ndependently of the keys. Even f an adversary can see your code, he or she cannot fnd a bad set of keys, snce he or she doesn t know exactly whch hash functon wll be chosen. What famly of functons should we select? Introducton to Algorthms September 29, 2004 L7.21
22 Famly of hash functons Idea #1: Take the famly of all functons h: U {0 m-1} That s, choose each of h(0), h(1),, h(u-1) ndependently at random from {0 m-1} Beneft: The unform hashng assumpton s true! Drawback: We need u random numbers to specfy h. Where to store them? Introducton to Algorthms September 29, 2004 L7.22
23 Unversal hashng Idea #2: Unversal Hashng Let H be a fnte collecton of hash functons, each mappng U to {0, 1,, m 1} We say H s unversal f for all x, y U, where x y, we have Pr h H {h(x) = h(y)} = 1/m. Introducton to Algorthms September 29, 2004 L7.23
24 Unversalty s good Theorem. Let h be a hash functon chosen (unformly) at random from a unversal set H of hash functons. Suppose h s used to hash n arbtrary keys nto the m slots of a table T. Then, for a gven key x, we have E[#collsons wth x] < n/m. Introducton to Algorthms September 29, 2004 L7.24
25 Proof of theorem Proof. Let C x be the random varable denotng the total number of collsons of keys n T wth x, and let 1 f h(x) = h(y), c xy = 0 otherwse. Note: E[c xy ] = 1/m and C =. x c xy y T {x} Introducton to Algorthms September 29, 2004 L7.25
26 Proof (contnued) E [ C ] = E y T x c xy { x} Take expectaton of both sdes. Introducton to Algorthms September 29, 2004 L7.26
27 Introducton to Algorthms September 29, 2004 L7.27 Proof (contnued) = = } { } { ] [ ] [ x T y xy x T y xy x c E c E C E Lnearty of expectaton. Take expectaton of both sdes.
28 Introducton to Algorthms September 29, 2004 L7.28 Proof (contnued) = = = } { } { } { 1/ ] [ ] [ x T y x T y xy x T y xy x m c E c E C E E[c xy ] = 1/m. Lnearty of expectaton. Take expectaton of both sdes.
29 Introducton to Algorthms September 29, 2004 L7.29 Proof (contnued) m n m c E c E C E x T y x T y xy x T y xy x 1 1/ ] [ ] [ } { } { } { = = = = Take expectaton of both sdes. Lnearty of expectaton. E[c xy ] = 1/m. Algebra..
30 Constructng a set of unversal hash functons Let m be prme. Decompose key k nto r + 1 dgts, each wth value n the set {0, 1,, m 1}. That s, let k = k 0, k 1,, k r, where 0 k < m. Randomzed strategy: Pck a = a 0, a 1,, a r where each a s chosen randomly from {0, 1,, m 1}. Defne h ( k) a = r = 0 a k mod m Denote H={h a :aas above} Introducton to Algorthms September 29, 2004 L7.30
31 Unversalty of dot-product hash functons Theorem. The set H = {h a } s unversal. Proof. Suppose that x = x 0, x 1,, x r and y = y 0, y 1,, y r are dstnct keys. Thus, they dffer n at least one dgt poston, wlog poston 0. What s the probablty that x and y collde, that s h a (x)=h b (y)? Introducton to Algorthms September 29, 2004 L7.31
32 Introducton to Algorthms September 29, 2004 L7.32 Proof (contnued) ) (mod 0 ) ( 0 m y x a r = ) (mod 0 ) ( ) ( m y x a y x a r + = ) (mod ) ( ) ( m y x a y x a r =. ) (mod ) ( ) ( 0 0 m y a x a b h x h r r a a = = =
33 Recall PS 2 Theorem. Let m be prme. For any z Z m such that z 0, there exsts a unque z 1 Z m such that z z 1 1 (mod m). Introducton to Algorthms September 29, 2004 L7.33
34 Introducton to Algorthms September 29, 2004 L7.34 Back to the proof ) (mod ) ( ) ( m y x a y x a r = We have and snce x 0 y 0, an nverse (x 0 y 0 ) 1 must exst, whch mples that, ) (mod ) ( ) ( m y x y x a a r =. Thus, for any choces of a 1, a 2,, a r, exactly one choce of a 0 causes x and y to collde.
35 Proof (completed) Q. What s the probablty that x and y collde? A. There are m choces for a 0, but exactly one choce for a 0 causes x and y to collde, namely r a a ( x y ) ( x y ) = mod m. = 1 Thus, the probablty of x and y colldng s 1/m. Introducton to Algorthms September 29, 2004 L7.35
36 Recap Showed how to mplement dctonary so that INSERT, DELETE, SEARCH work n expected constant tme under the unform hashng assumpton Relaxed the assumpton to unversal hashng Constructed unversal hashng for keys n {0 m r -1} Introducton to Algorthms September 29, 2004 L7.36
37 Perfect hashng Gven a set of n keys, construct a statc hash table of sze m = O(n) such that SEARCH takes Θ(1) tme n the worst case. IDEA: Twolevel scheme wth unversal 2 hashng at 3 both levels No collsons at level 2! T S S h 31 (14) = h 31 (27) = 1 4 S m a Introducton to Algorthms September 29, 2004 L7.37
38 Collsons at level 2 Theorem. Let H be a class of unversal hash functons for a table of sze m = n 2. Then, f we use a random h H to hash n keys nto the table, the expected number of collsons s at most 1/2. Proof. By the defnton of unversalty, the probablty that 2 gven keys n the table collde under h s 1/m = 1/n 2. Snce there are ( n) pars 2 of keys that can possbly collde, the expected number of collsons s n 1 n( n 1) = n n < Introducton to Algorthms September 29, 2004 L
39 No collsons at level 2 Corollary. The probablty of no collsons s at least 1/2. Proof. Markov s nequalty says that for any nonnegatve random varable X, we have Pr{X t} E[X]/t. Applyng ths nequalty wth t = 1, we fnd that the probablty of 1 or more collsons s at most 1/2. Thus, just by testng random hash functons n H, we ll quckly fnd one that works. Introducton to Algorthms September 29, 2004 L7.39
40 Analyss of storage For the level-1 hash table T, choose m = n, and let n be random varable for the number of keys that hash to slot n T. By usng n 2 slots for the level-2 hash table S, the expected total storage requred for the two-level scheme s therefore E m 1 = 0 ( n 2 ) Θ = Θ( n), snce the analyss s dentcal to the analyss from rectaton of the expected runnng tme of bucket sort. (For a probablty bound, apply Markov.) Introducton to Algorthms September 29, 2004 L7.40
41 Resolvng collsons by open addressng No storage s used outsde of the hash table tself. Inserton systematcally probes the table untl an empty slot s found. The hash functon depends on both the key and probe number: h : U {0, 1,, m 1} {0, 1,, m 1}. The probe sequence h(k,0), h(k,1),, h(k,m 1) should be a permutaton of {0, 1,, m 1}. The table may fll up, and deleton s dffcult (but not mpossble). Introducton to Algorthms September 29, 2004 L7.41
42 Example of open addressng Insert key k = 496: 0. Probe h(496,0) T collson m 1 Introducton to Algorthms September 29, 2004 L7.42
43 Example of open addressng Insert key k = 496: T 0. Probe h(496,0) 1. Probe h(496,1) 586 collson m 1 Introducton to Algorthms September 29, 2004 L7.43
44 Example of open addressng Insert key k = 496: 0. Probe h(496,0) 1. Probe h(496,1) 2. Probe h(496,2) T nserton m 1 Introducton to Algorthms September 29, 2004 L7.44
45 Example of open addressng Search for key k = 496: 0. Probe h(496,0) 1. Probe h(496,1) 2. Probe h(496,2) T Search uses the same probe sequence, termnatng successfully f t fnds the key m 1 and unsuccessfully f t encounters an empty slot. 0 Introducton to Algorthms September 29, 2004 L7.45
46 Probng strateges Lnear probng: Gven an ordnary hash functon h (k), lnear probng uses the hash functon h(k,) = (h (k) + ) mod m. Ths method, though smple, suffers from prmary clusterng, where long runs of occuped slots buld up, ncreasng the average search tme. Moreover, the long runs of occuped slots tend to get longer. Introducton to Algorthms September 29, 2004 L7.46
47 Probng strateges Double hashng Gven two ordnary hash functons h 1 (k) and h 2 (k), double hashng uses the hash functon h(k,) = (h 1 (k) + h 2 (k)) mod m. Ths method generally produces excellent results, but h 2 (k) must be relatvely prme to m. One way s to make m a power of 2 and desgn h 2 (k) to produce only odd numbers. Introducton to Algorthms September 29, 2004 L7.47
48 Analyss of open addressng We make the assumpton of unform hashng: Each key s equally lkely to have any one of the m! permutatons as ts probe sequence. Theorem. Gven an open-addressed hash table wth load factor α = n/m < 1, the expected number of probes n an unsuccessful search s at most 1/(1 α). Introducton to Algorthms September 29, 2004 L7.48
49 Proof of the theorem Proof. At least one probe s always necessary. Wth probablty n/m, the frst probe hts an occuped slot, and a second probe s necessary. Wth probablty (n 1)/(m 1), the second probe hts an occuped slot, and a thrd probe s necessary. Wth probablty (n 2)/(m 2), the thrd probe hts an occuped slot, etc. Observe that n m < n m = α for = 1, 2,, n. Introducton to Algorthms September 29, 2004 L7.49
50 Introducton to Algorthms September 29, 2004 L7.50 Proof (contnued) Therefore, the expected number of probes s L L n m m n m n m n ( ) ( ) ( ) ( ) α α α α α α α α α = = = L L L. The textbook has a more rgorous proof.
51 Implcatons of the theorem If α s constant, then accessng an openaddressed hash table takes constant tme. If the table s half full, then the expected number of probes s 1/(1 0.5) = 2. If the table s 90% full, then the expected number of probes s 1/(1 0.9) = 10. Introducton to Algorthms September 29, 2004 L7.51
52 Dot-product method Randomzed strategy: Let m be prme. Decompose key k nto r + 1 dgts, each wth value n the set {0, 1,, m 1}. That s, let k = k 0, k 1,, k m 1, where 0 k < m. Pck a = a 0, a 1,, a m 1 where each a s chosen randomly from {0, 1,, m 1}. r = 0 Defne h ( k) = a k mod m. a Excellent n practce, but expensve to compute. Introducton to Algorthms September 29, 2004 L7.52
Introduction to Algorithms
Introducton to Algorthms 6.046J/8.40J Lecture 7 Prof. Potr Indyk Data Structures Role of data structures: Encapsulate data Support certan operatons (e.g., INSERT, DELETE, SEARCH) Our focus: effcency of
More informationHashing. Alexandra Stefan
Hashng Alexandra Stefan 1 Hash tables Tables Drect access table (or key-ndex table): key => ndex Hash table: key => hash value => ndex Man components Hash functon Collson resoluton Dfferent keys mapped
More informationHashing, Hash Functions. Lecture 7
Hashing, Hash Functions Lecture 7 Symbol-table problem Symbol table T holding n records: x record key[x] Other fields containing satellite data Operations on T: INSERT(T, x) DELETE(T, x) SEARCH(T, k) How
More informationAlgorithms Design & Analysis. Hash Tables
Algorthms Desg & Aalyss Hash Tables Recap Lower boud Order statstcs 2 Today s topcs Drect-accessble table Hash tables Hash fuctos Uversal hashg Perfect Hashg Ope addressg 3 Symbol-table problem Symbol
More informationLecture 3 January 31, 2017
CS 224: Advanced Algorthms Sprng 207 Prof. Jelan Nelson Lecture 3 January 3, 207 Scrbe: Saketh Rama Overvew In the last lecture we covered Y-fast tres and Fuson Trees. In ths lecture we start our dscusson
More informationLecture: Analysis of Algorithms (CS )
Lecture: Analysis of Algorithms (CS483-001) Amarda Shehu Spring 2017 1 Outline of Today s Class 2 Choosing Hash Functions Universal Universality Theorem Constructing a Set of Universal Hash Functions Perfect
More informationData Structures and Algorithm. Xiaoqing Zheng
Data Structures and Algorithm Xiaoqing Zheng zhengxq@fudan.edu.cn Dictionary problem Dictionary T holding n records: x records key[x] Other fields containing satellite data Operations on T: INSERT(T, x)
More informationSome Consequences. Example of Extended Euclidean Algorithm. The Fundamental Theorem of Arithmetic, II. Characterizing the GCD and LCM
Example of Extended Eucldean Algorthm Recall that gcd(84, 33) = gcd(33, 18) = gcd(18, 15) = gcd(15, 3) = gcd(3, 0) = 3 We work backwards to wrte 3 as a lnear combnaton of 84 and 33: 3 = 18 15 [Now 3 s
More informationLecture 4: Universal Hash Functions/Streaming Cont d
CSE 5: Desgn and Analyss of Algorthms I Sprng 06 Lecture 4: Unversal Hash Functons/Streamng Cont d Lecturer: Shayan Oves Gharan Aprl 6th Scrbe: Jacob Schreber Dsclamer: These notes have not been subjected
More informationSymbol-table problem. Hashing. Direct-access table. Hash functions. CS Spring Symbol table T holding n records: record.
CS 5633 -- Spring 25 Symbol-table problem Hashing Carola Wenk Slides courtesy of Charles Leiserson with small changes by Carola Wenk CS 5633 Analysis of Algorithms 1 Symbol table holding n records: record
More informationStanford University CS359G: Graph Partitioning and Expanders Handout 4 Luca Trevisan January 13, 2011
Stanford Unversty CS359G: Graph Parttonng and Expanders Handout 4 Luca Trevsan January 3, 0 Lecture 4 In whch we prove the dffcult drecton of Cheeger s nequalty. As n the past lectures, consder an undrected
More informationTHE CHINESE REMAINDER THEOREM. We should thank the Chinese for their wonderful remainder theorem. Glenn Stevens
THE CHINESE REMAINDER THEOREM KEITH CONRAD We should thank the Chnese for ther wonderful remander theorem. Glenn Stevens 1. Introducton The Chnese remander theorem says we can unquely solve any par of
More informationExpected Value and Variance
MATH 38 Expected Value and Varance Dr. Neal, WKU We now shall dscuss how to fnd the average and standard devaton of a random varable X. Expected Value Defnton. The expected value (or average value, or
More informationHash functions : MAC / HMAC
Hash functons : MAC / HMAC Outlne Message Authentcaton Codes Keyed hash famly Uncondtonally Secure MACs Ref: D Stnson: Cryprography Theory and Practce (3 rd ed), Chap 4. Unversal hash famly Notatons: X
More informationprinceton univ. F 13 cos 521: Advanced Algorithm Design Lecture 3: Large deviations bounds and applications Lecturer: Sanjeev Arora
prnceton unv. F 13 cos 521: Advanced Algorthm Desgn Lecture 3: Large devatons bounds and applcatons Lecturer: Sanjeev Arora Scrbe: Today s topc s devaton bounds: what s the probablty that a random varable
More information04 - Treaps. Dr. Alexander Souza
Algorths Theory 04 - Treaps Dr. Alexander Souza The dctonary proble Gven: Unverse (U,
More informationCHAPTER 17 Amortized Analysis
CHAPTER 7 Amortzed Analyss In an amortzed analyss, the tme requred to perform a sequence of data structure operatons s averaged over all the operatons performed. It can be used to show that the average
More informationDirichlet s Theorem In Arithmetic Progressions
Drchlet s Theorem In Arthmetc Progressons Parsa Kavkan Hang Wang The Unversty of Adelade February 26, 205 Abstract The am of ths paper s to ntroduce and prove Drchlet s theorem n arthmetc progressons,
More informationExample: (13320, 22140) =? Solution #1: The divisors of are 1, 2, 3, 4, 5, 6, 9, 10, 12, 15, 18, 20, 27, 30, 36, 41,
The greatest common dvsor of two ntegers a and b (not both zero) s the largest nteger whch s a common factor of both a and b. We denote ths number by gcd(a, b), or smply (a, b) when there s no confuson
More informationMATH 5707 HOMEWORK 4 SOLUTIONS 2. 2 i 2p i E(X i ) + E(Xi 2 ) ä i=1. i=1
MATH 5707 HOMEWORK 4 SOLUTIONS CİHAN BAHRAN 1. Let v 1,..., v n R m, all lengths v are not larger than 1. Let p 1,..., p n [0, 1] be arbtrary and set w = p 1 v 1 + + p n v n. Then there exst ε 1,..., ε
More informationFoundations of Arithmetic
Foundatons of Arthmetc Notaton We shall denote the sum and product of numbers n the usual notaton as a 2 + a 2 + a 3 + + a = a, a 1 a 2 a 3 a = a The notaton a b means a dvdes b,.e. ac = b where c s an
More information18.1 Introduction and Recap
CS787: Advanced Algorthms Scrbe: Pryananda Shenoy and Shjn Kong Lecturer: Shuch Chawla Topc: Streamng Algorthmscontnued) Date: 0/26/2007 We contnue talng about streamng algorthms n ths lecture, ncludng
More informationStanford University CS254: Computational Complexity Notes 7 Luca Trevisan January 29, Notes for Lecture 7
Stanford Unversty CS54: Computatonal Complexty Notes 7 Luca Trevsan January 9, 014 Notes for Lecture 7 1 Approxmate Countng wt an N oracle We complete te proof of te followng result: Teorem 1 For every
More informationLectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix
Lectures - Week 4 Matrx norms, Condtonng, Vector Spaces, Lnear Independence, Spannng sets and Bass, Null space and Range of a Matrx Matrx Norms Now we turn to assocatng a number to each matrx. We could
More informationMining Data Streams-Estimating Frequency Moment
Mnng Data Streams-Estmatng Frequency Moment Barna Saha October 26, 2017 Frequency Moment Computng moments nvolves dstrbuton of frequences of dfferent elements n the stream. Frequency Moment Computng moments
More informationMin Cut, Fast Cut, Polynomial Identities
Randomzed Algorthms, Summer 016 Mn Cut, Fast Cut, Polynomal Identtes Instructor: Thomas Kesselhem and Kurt Mehlhorn 1 Mn Cuts n Graphs Lecture (5 pages) Throughout ths secton, G = (V, E) s a mult-graph.
More informationThe Multiple Classical Linear Regression Model (CLRM): Specification and Assumptions. 1. Introduction
ECONOMICS 5* -- NOTE (Summary) ECON 5* -- NOTE The Multple Classcal Lnear Regresson Model (CLRM): Specfcaton and Assumptons. Introducton CLRM stands for the Classcal Lnear Regresson Model. The CLRM s also
More informationInner Product. Euclidean Space. Orthonormal Basis. Orthogonal
Inner Product Defnton 1 () A Eucldean space s a fnte-dmensonal vector space over the reals R, wth an nner product,. Defnton 2 (Inner Product) An nner product, on a real vector space X s a symmetrc, blnear,
More informationAPPENDIX A Some Linear Algebra
APPENDIX A Some Lnear Algebra The collecton of m, n matrces A.1 Matrces a 1,1,..., a 1,n A = a m,1,..., a m,n wth real elements a,j s denoted by R m,n. If n = 1 then A s called a column vector. Smlarly,
More informationLecture 3. Ax x i a i. i i
18.409 The Behavor of Algorthms n Practce 2/14/2 Lecturer: Dan Spelman Lecture 3 Scrbe: Arvnd Sankar 1 Largest sngular value In order to bound the condton number, we need an upper bound on the largest
More informationU.C. Berkeley CS294: Spectral Methods and Expanders Handout 8 Luca Trevisan February 17, 2016
U.C. Berkeley CS94: Spectral Methods and Expanders Handout 8 Luca Trevsan February 7, 06 Lecture 8: Spectral Algorthms Wrap-up In whch we talk about even more generalzatons of Cheeger s nequaltes, and
More informationU.C. Berkeley CS294: Beyond Worst-Case Analysis Luca Trevisan September 5, 2017
U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 4s Luca Trevsan September 5, 07 Summary of Lecture 4 In whch we ntroduce semdefnte programmng and apply t to Max Cut. Semdefnte Programmng Recall that
More informationE Tail Inequalities. E.1 Markov s Inequality. Non-Lecture E: Tail Inequalities
Algorthms Non-Lecture E: Tal Inequaltes If you hold a cat by the tal you learn thngs you cannot learn any other way. Mar Twan E Tal Inequaltes The smple recursve structure of sp lsts made t relatvely easy
More informationCalculation of time complexity (3%)
Problem 1. (30%) Calculaton of tme complexty (3%) Gven n ctes, usng exhaust search to see every result takes O(n!). Calculaton of tme needed to solve the problem (2%) 40 ctes:40! dfferent tours 40 add
More information11 Tail Inequalities Markov s Inequality. Lecture 11: Tail Inequalities [Fa 13]
Algorthms Lecture 11: Tal Inequaltes [Fa 13] If you hold a cat by the tal you learn thngs you cannot learn any other way. Mark Twan 11 Tal Inequaltes The smple recursve structure of skp lsts made t relatvely
More informationWeek 2. This week, we covered operations on sets and cardinality.
Week 2 Ths week, we covered operatons on sets and cardnalty. Defnton 0.1 (Correspondence). A correspondence between two sets A and B s a set S contaned n A B = {(a, b) a A, b B}. A correspondence from
More informationfind (x): given element x, return the canonical element of the set containing x;
COS 43 Sprng, 009 Dsjont Set Unon Problem: Mantan a collecton of dsjont sets. Two operatons: fnd the set contanng a gven element; unte two sets nto one (destructvely). Approach: Canoncal element method:
More informationDISCRIMINANTS AND RAMIFIED PRIMES. 1. Introduction A prime number p is said to be ramified in a number field K if the prime ideal factorization
DISCRIMINANTS AND RAMIFIED PRIMES KEITH CONRAD 1. Introducton A prme number p s sad to be ramfed n a number feld K f the prme deal factorzaton (1.1) (p) = po K = p e 1 1 peg g has some e greater than 1.
More informationEstimation: Part 2. Chapter GREG estimation
Chapter 9 Estmaton: Part 2 9. GREG estmaton In Chapter 8, we have seen that the regresson estmator s an effcent estmator when there s a lnear relatonshp between y and x. In ths chapter, we generalzed the
More informationMath 261 Exercise sheet 2
Math 261 Exercse sheet 2 http://staff.aub.edu.lb/~nm116/teachng/2017/math261/ndex.html Verson: September 25, 2017 Answers are due for Monday 25 September, 11AM. The use of calculators s allowed. Exercse
More informationFinding Primitive Roots Pseudo-Deterministically
Electronc Colloquum on Computatonal Complexty, Report No 207 (205) Fndng Prmtve Roots Pseudo-Determnstcally Ofer Grossman December 22, 205 Abstract Pseudo-determnstc algorthms are randomzed search algorthms
More informationOn cyclic of Steiner system (v); V=2,3,5,7,11,13
On cyclc of Stener system (v); V=,3,5,7,,3 Prof. Dr. Adl M. Ahmed Rana A. Ibraham Abstract: A stener system can be defned by the trple S(t,k,v), where every block B, (=,,,b) contans exactly K-elementes
More informationAlgorithms for factoring
CSA E0 235: Crytograhy Arl 9,2015 Instructor: Arta Patra Algorthms for factorng Submtted by: Jay Oza, Nranjan Sngh Introducton Factorsaton of large ntegers has been a wdely studed toc manly because of
More informationNotes on Frequency Estimation in Data Streams
Notes on Frequency Estmaton n Data Streams In (one of) the data streamng model(s), the data s a sequence of arrvals a 1, a 2,..., a m of the form a j = (, v) where s the dentty of the tem and belongs to
More informationModule 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Verson ECE IIT, Kharagpur Lesson 6 Theory of Quantzaton Verson ECE IIT, Kharagpur Instructonal Objectves At the end of ths lesson, the students should be able to:
More informationCase A. P k = Ni ( 2L i k 1 ) + (# big cells) 10d 2 P k.
THE CELLULAR METHOD In ths lecture, we ntroduce the cellular method as an approach to ncdence geometry theorems lke the Szemeréd-Trotter theorem. The method was ntroduced n the paper Combnatoral complexty
More informationProvable Security Signatures
Provable Securty Sgnatures UCL - Louvan-la-Neuve Wednesday, July 10th, 2002 LIENS-CNRS Ecole normale supéreure Summary Introducton Sgnature FD PSS Forkng Lemma Generc Model Concluson Provable Securty -
More informationHash tables. Hash tables
Basic Probability Theory Two events A, B are independent if Conditional probability: Pr[A B] = Pr[A] Pr[B] Pr[A B] = Pr[A B] Pr[B] The expectation of a (discrete) random variable X is E[X ] = k k Pr[X
More informationLecture 10: May 6, 2013
TTIC/CMSC 31150 Mathematcal Toolkt Sprng 013 Madhur Tulsan Lecture 10: May 6, 013 Scrbe: Wenje Luo In today s lecture, we manly talked about random walk on graphs and ntroduce the concept of graph expander,
More informationProblem Solving in Math (Math 43900) Fall 2013
Problem Solvng n Math (Math 43900) Fall 2013 Week four (September 17) solutons Instructor: Davd Galvn 1. Let a and b be two nteger for whch a b s dvsble by 3. Prove that a 3 b 3 s dvsble by 9. Soluton:
More informationLecture Space-Bounded Derandomization
Notes on Complexty Theory Last updated: October, 2008 Jonathan Katz Lecture Space-Bounded Derandomzaton 1 Space-Bounded Derandomzaton We now dscuss derandomzaton of space-bounded algorthms. Here non-trval
More informationSL n (F ) Equals its Own Derived Group
Internatonal Journal of Algebra, Vol. 2, 2008, no. 12, 585-594 SL n (F ) Equals ts Own Derved Group Jorge Macel BMCC-The Cty Unversty of New York, CUNY 199 Chambers street, New York, NY 10007, USA macel@cms.nyu.edu
More informationChapter 8 SCALAR QUANTIZATION
Outlne Chapter 8 SCALAR QUANTIZATION Yeuan-Kuen Lee [ CU, CSIE ] 8.1 Overvew 8. Introducton 8.4 Unform Quantzer 8.5 Adaptve Quantzaton 8.6 Nonunform Quantzaton 8.7 Entropy-Coded Quantzaton Ch 8 Scalar
More informationRandomness and Computation
Randomness and Computaton or, Randomzed Algorthms Mary Cryan School of Informatcs Unversty of Ednburgh RC 208/9) Lecture 0 slde Balls n Bns m balls, n bns, and balls thrown unformly at random nto bns usually
More informationEconomics 130. Lecture 4 Simple Linear Regression Continued
Economcs 130 Lecture 4 Contnued Readngs for Week 4 Text, Chapter and 3. We contnue wth addressng our second ssue + add n how we evaluate these relatonshps: Where do we get data to do ths analyss? How do
More informationFACTORIZATION IN KRULL MONOIDS WITH INFINITE CLASS GROUP
C O L L O Q U I U M M A T H E M A T I C U M VOL. 80 1999 NO. 1 FACTORIZATION IN KRULL MONOIDS WITH INFINITE CLASS GROUP BY FLORIAN K A I N R A T H (GRAZ) Abstract. Let H be a Krull monod wth nfnte class
More informationProblem Set 9 Solutions
Desgn and Analyss of Algorthms May 4, 2015 Massachusetts Insttute of Technology 6.046J/18.410J Profs. Erk Demane, Srn Devadas, and Nancy Lynch Problem Set 9 Solutons Problem Set 9 Solutons Ths problem
More information6.842 Randomness and Computation February 18, Lecture 4
6.842 Randomness and Computaton February 18, 2014 Lecture 4 Lecturer: Rontt Rubnfeld Scrbe: Amartya Shankha Bswas Topcs 2-Pont Samplng Interactve Proofs Publc cons vs Prvate cons 1 Two Pont Samplng 1.1
More informationMaximizing the number of nonnegative subsets
Maxmzng the number of nonnegatve subsets Noga Alon Hao Huang December 1, 213 Abstract Gven a set of n real numbers, f the sum of elements of every subset of sze larger than k s negatve, what s the maxmum
More informationDifference Equations
Dfference Equatons c Jan Vrbk 1 Bascs Suppose a sequence of numbers, say a 0,a 1,a,a 3,... s defned by a certan general relatonshp between, say, three consecutve values of the sequence, e.g. a + +3a +1
More information= z 20 z n. (k 20) + 4 z k = 4
Problem Set #7 solutons 7.2.. (a Fnd the coeffcent of z k n (z + z 5 + z 6 + z 7 + 5, k 20. We use the known seres expanson ( n+l ( z l l z n below: (z + z 5 + z 6 + z 7 + 5 (z 5 ( + z + z 2 + z + 5 5
More information1. Inference on Regression Parameters a. Finding Mean, s.d and covariance amongst estimates. 2. Confidence Intervals and Working Hotelling Bands
Content. Inference on Regresson Parameters a. Fndng Mean, s.d and covarance amongst estmates.. Confdence Intervals and Workng Hotellng Bands 3. Cochran s Theorem 4. General Lnear Testng 5. Measures of
More informationMath Review. CptS 223 Advanced Data Structures. Larry Holder School of Electrical Engineering and Computer Science Washington State University
Math Revew CptS 223 dvanced Data Structures Larry Holder School of Electrcal Engneerng and Computer Scence Washngton State Unversty 1 Why do we need math n a data structures course? nalyzng data structures
More informationFinding Dense Subgraphs in G(n, 1/2)
Fndng Dense Subgraphs n Gn, 1/ Atsh Das Sarma 1, Amt Deshpande, and Rav Kannan 1 Georga Insttute of Technology,atsh@cc.gatech.edu Mcrosoft Research-Bangalore,amtdesh,annan@mcrosoft.com Abstract. Fndng
More informationAnti-van der Waerden numbers of 3-term arithmetic progressions.
Ant-van der Waerden numbers of 3-term arthmetc progressons. Zhanar Berkkyzy, Alex Schulte, and Mchael Young Aprl 24, 2016 Abstract The ant-van der Waerden number, denoted by aw([n], k), s the smallest
More information1 Generating functions, continued
Generatng functons, contnued. Generatng functons and parttons We can make use of generatng functons to answer some questons a bt more restrctve than we ve done so far: Queston : Fnd a generatng functon
More informationProf. Dr. I. Nasser Phys 630, T Aug-15 One_dimensional_Ising_Model
EXACT OE-DIMESIOAL ISIG MODEL The one-dmensonal Isng model conssts of a chan of spns, each spn nteractng only wth ts two nearest neghbors. The smple Isng problem n one dmenson can be solved drectly n several
More informationHomework 9 Solutions. 1. (Exercises from the book, 6 th edition, 6.6, 1-3.) Determine the number of distinct orderings of the letters given:
Homework 9 Solutons PROBLEM ONE 1 (Exercses from the book, th edton,, 1-) Determne the number of dstnct orderngs of the letters gven: (a) GUIDE Soluton: 5! (b) SCHOOL Soluton:! (c) SALESPERSONS Soluton:
More informationFirst Year Examination Department of Statistics, University of Florida
Frst Year Examnaton Department of Statstcs, Unversty of Florda May 7, 010, 8:00 am - 1:00 noon Instructons: 1. You have four hours to answer questons n ths examnaton.. You must show your work to receve
More informationIntroductory Cardinality Theory Alan Kaylor Cline
Introductory Cardnalty Theory lan Kaylor Clne lthough by name the theory of set cardnalty may seem to be an offshoot of combnatorcs, the central nterest s actually nfnte sets. Combnatorcs deals wth fnte
More informationLecture 5 Decoding Binary BCH Codes
Lecture 5 Decodng Bnary BCH Codes In ths class, we wll ntroduce dfferent methods for decodng BCH codes 51 Decodng the [15, 7, 5] 2 -BCH Code Consder the [15, 7, 5] 2 -code C we ntroduced n the last lecture
More informationHMMT February 2016 February 20, 2016
HMMT February 016 February 0, 016 Combnatorcs 1. For postve ntegers n, let S n be the set of ntegers x such that n dstnct lnes, no three concurrent, can dvde a plane nto x regons (for example, S = {3,
More informationChapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems
Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons
More informationMarkov Chain Monte Carlo (MCMC), Gibbs Sampling, Metropolis Algorithms, and Simulated Annealing Bioinformatics Course Supplement
Markov Chan Monte Carlo MCMC, Gbbs Samplng, Metropols Algorthms, and Smulated Annealng 2001 Bonformatcs Course Supplement SNU Bontellgence Lab http://bsnuackr/ Outlne! Markov Chan Monte Carlo MCMC! Metropols-Hastngs
More informationExercises of Chapter 2
Exercses of Chapter Chuang-Cheh Ln Department of Computer Scence and Informaton Engneerng, Natonal Chung Cheng Unversty, Mng-Hsung, Chay 61, Tawan. Exercse.6. Suppose that we ndependently roll two standard
More informationHash tables. Hash tables
Dictionary Definition A dictionary is a data-structure that stores a set of elements where each element has a unique key, and supports the following operations: Search(S, k) Return the element whose key
More information3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X
Statstcs 1: Probablty Theory II 37 3 EPECTATION OF SEVERAL RANDOM VARIABLES As n Probablty Theory I, the nterest n most stuatons les not on the actual dstrbuton of a random vector, but rather on a number
More informationThe Geometry of Logit and Probit
The Geometry of Logt and Probt Ths short note s meant as a supplement to Chapters and 3 of Spatal Models of Parlamentary Votng and the notaton and reference to fgures n the text below s to those two chapters.
More informationSociété de Calcul Mathématique SA
Socété de Calcul Mathématque SA Outls d'ade à la décson Tools for decson help Probablstc Studes: Normalzng the Hstograms Bernard Beauzamy December, 202 I. General constructon of the hstogram Any probablstc
More informationInternational Mathematical Olympiad. Preliminary Selection Contest 2012 Hong Kong. Outline of Solutions
Internatonal Mathematcal Olympad Prelmnary Selecton ontest Hong Kong Outlne of Solutons nswers: 7 4 7 4 6 5 9 6 99 7 6 6 9 5544 49 5 7 4 6765 5 6 6 7 6 944 9 Solutons: Snce n s a two-dgt number, we have
More informationAlgebraic properties of polynomial iterates
Algebrac propertes of polynomal terates Alna Ostafe Department of Computng Macquare Unversty 1 Motvaton 1. Better and cryptographcally stronger pseudorandom number generators (PRNG) as lnear constructons
More informationLecture 4: November 17, Part 1 Single Buffer Management
Lecturer: Ad Rosén Algorthms for the anagement of Networs Fall 2003-2004 Lecture 4: November 7, 2003 Scrbe: Guy Grebla Part Sngle Buffer anagement In the prevous lecture we taled about the Combned Input
More informationCOS 511: Theoretical Machine Learning
COS 5: Theoretcal Machne Learnng Lecturer: Rob Schapre Lecture #0 Scrbe: José Sões Ferrera March 06, 203 In the last lecture the concept of Radeacher coplexty was ntroduced, wth the goal of showng that
More informationSmarandache-Zero Divisors in Group Rings
Smarandache-Zero Dvsors n Group Rngs W.B. Vasantha and Moon K. Chetry Department of Mathematcs I.I.T Madras, Chenna The study of zero-dvsors n group rngs had become nterestng problem snce 1940 wth the
More informationSupplement: Proofs and Technical Details for The Solution Path of the Generalized Lasso
Supplement: Proofs and Techncal Detals for The Soluton Path of the Generalzed Lasso Ryan J. Tbshran Jonathan Taylor In ths document we gve supplementary detals to the paper The Soluton Path of the Generalzed
More informationMatrix Approximation via Sampling, Subspace Embedding. 1 Solving Linear Systems Using SVD
Matrx Approxmaton va Samplng, Subspace Embeddng Lecturer: Anup Rao Scrbe: Rashth Sharma, Peng Zhang 0/01/016 1 Solvng Lnear Systems Usng SVD Two applcatons of SVD have been covered so far. Today we loo
More informationn α j x j = 0 j=1 has a nontrivial solution. Here A is the n k matrix whose jth column is the vector for all t j=0
MODULE 2 Topcs: Lnear ndependence, bass and dmenson We have seen that f n a set of vectors one vector s a lnear combnaton of the remanng vectors n the set then the span of the set s unchanged f that vector
More informationLearning Theory: Lecture Notes
Learnng Theory: Lecture Notes Lecturer: Kamalka Chaudhur Scrbe: Qush Wang October 27, 2012 1 The Agnostc PAC Model Recall that one of the constrants of the PAC model s that the data dstrbuton has to be
More informationErrors for Linear Systems
Errors for Lnear Systems When we solve a lnear system Ax b we often do not know A and b exactly, but have only approxmatons  and ˆb avalable. Then the best thng we can do s to solve ˆx ˆb exactly whch
More informationAppendix B: Resampling Algorithms
407 Appendx B: Resamplng Algorthms A common problem of all partcle flters s the degeneracy of weghts, whch conssts of the unbounded ncrease of the varance of the mportance weghts ω [ ] of the partcles
More informationHash tables. Hash tables
Dictionary Definition A dictionary is a data-structure that stores a set of elements where each element has a unique key, and supports the following operations: Search(S, k) Return the element whose key
More informationLecture Notes on Linear Regression
Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume
More informationA 2D Bounded Linear Program (H,c) 2D Linear Programming
A 2D Bounded Lnear Program (H,c) h 3 v h 8 h 5 c h 4 h h 6 h 7 h 2 2D Lnear Programmng C s a polygonal regon, the ntersecton of n halfplanes. (H, c) s nfeasble, as C s empty. Feasble regon C s unbounded
More information18.781: Solution to Practice Questions for Final Exam
18.781: Soluton to Practce Questons for Fnal Exam 1. Fnd three solutons n postve ntegers of x 6y = 1 by frst calculatng the contnued fracton expanson of 6. Soluton: We have 1 6=[, ] 6 6+ =[, ] 1 =[,, ]=[,,
More informationTHE SUMMATION NOTATION Ʃ
Sngle Subscrpt otaton THE SUMMATIO OTATIO Ʃ Most of the calculatons we perform n statstcs are repettve operatons on lsts of numbers. For example, we compute the sum of a set of numbers, or the sum of the
More informationU.C. Berkeley CS294: Beyond Worst-Case Analysis Handout 6 Luca Trevisan September 12, 2017
U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 6 Luca Trevsan September, 07 Scrbed by Theo McKenze Lecture 6 In whch we study the spectrum of random graphs. Overvew When attemptng to fnd n polynomal
More informationLecture 3: Probability Distributions
Lecture 3: Probablty Dstrbutons Random Varables Let us begn by defnng a sample space as a set of outcomes from an experment. We denote ths by S. A random varable s a functon whch maps outcomes nto the
More informationP exp(tx) = 1 + t 2k M 2k. k N
1. Subgaussan tals Defnton. Say that a random varable X has a subgaussan dstrbuton wth scale factor σ< f P exp(tx) exp(σ 2 t 2 /2) for all real t. For example, f X s dstrbuted N(,σ 2 ) then t s subgaussan.
More informationTornado and Luby Transform Codes. Ashish Khisti Presentation October 22, 2003
Tornado and Luby Transform Codes Ashsh Khst 6.454 Presentaton October 22, 2003 Background: Erasure Channel Elas[956] studed the Erasure Channel β x x β β x 2 m x 2 k? Capacty of Noseless Erasure Channel
More informationDiscussion 11 Summary 11/20/2018
Dscusson 11 Summary 11/20/2018 1 Quz 8 1. Prove for any sets A, B that A = A B ff B A. Soluton: There are two drectons we need to prove: (a) A = A B B A, (b) B A A = A B. (a) Frst, we prove A = A B B A.
More information