A Master Theorem for Discrete Divide and Conquer Recurrences
|
|
- Emory Greene
- 5 years ago
- Views:
Transcription
1 A Master Theorem for Discrete Divide and Conquer Recurrences Wojciech Szpankowski Department of Computer Science Purdue University W. Lafayette, IN January 20, 2011 NSF CSoI SODA, 2011 Research supported by NSF Science & Technology Center, and Humboldt Foundation.
2 Outline 1. Divide and Conquer 2. Example: Boncelet s Algorithm 3. Continuous Relaxation of the Recurrence 4. Master Theorem 5. Examples 6. Sketch of Proof.
3 Divide and Conquer Divide and Conquer: A divide and conquer algorithm splits the input into several smaller subproblems, solving each subproblem separately, and then knitting together to solve the original problem. Complexity: A problem of size n is divided into m 2 subproblems of size p j n + δ j and p j n+δ j and each subproblem contributesb j,b j fraction to the final solution; there is a cost a n associated with combining subproblems. Total Cost: The total cost T(n) satisfies the discrete divide and conquer recurrence: T(n) = a n + m b j T ( p j n + δ j ) + m ) b j ( p T j n + δ j (n 2) j=1 j=1 where 0 p j < 1 (e.g., m i=1 p i = 1).
4 Example: Boncelet s Algorithm Boncelet s algorithm is a variable-to-fixed data compression scheme: 1. A variable-to-fixed length encoder partitions a source string over an m- ary alphabet into variable-length phrases. 2. Each phrase belongs to a given dictionary. 3. A dictionary is represented by a complete parsing tree. 4. The dictionary entries correspond to the leaves of the parsing tree.
5 Example: Boncelet s Algorithm Boncelet s algorithm is a variable-to-fixed data compression scheme: 1. A variable-to-fixed length encoder partitions a source string over an m- ary alphabet into variable-length phrases. 2. Each phrase belongs to a given dictionary. 3. A dictionary is represented by a complete parsing tree. 4. The dictionary entries correspond to the leaves of the parsing tree. Example: A binary string (m = 2) with symbol probabilities p 1 and p 2. The expected phrase length d(n) satisfies: d(n) = 1 + p 1 d( p 1 n + δ ) + p 2 d( p 2 n δ )
6 Continuous Relaxation We relax the discrete nature of the recurrence and consider a continuous version: m T(x) = a(x) + b j T(p j x)), x > 1, b j = 0. j=1 Akra and Bazzi (1998) proved that T(x) = Θ ( ( x x s where s 0 is a unique real root of j b jp j s 0 = 1. )) a(u) u s 0 +1du
7 Continuous Relaxation We relax the discrete nature of the recurrence and consider a continuous version: m T(x) = a(x) + b j T(p j x)), x > 1, b j = 0. j=1 Akra and Bazzi (1998) proved that ( T(x) = Θ x s 0 ( x where s 0 is a unique real root of j b jp j s 0 = 1. )) a(u) u s 0 +1du Indeed, by taking Mellin transform of the relaxed recurrence: t(s) = 0 T(x)x s 1 dx we find (for some a(s) and g(s)) t(s) = a(s) + g(s) 1 m j=1 b jp s. i An application of the Wiener-Ikehara theorem leads to T(x) Cx s 0 with C = a( s 0) + g( s 0 ) j b jp s 0 j log(1/p j ).
8 Discrete Divide & Conquer Recurrence by Dirichlet Series For a sequence c(n) define the Dirichlet series as C(s) = n=1 c(n) n s provided it exists for R(s) > σ c for some σ c. Theorem 1 (Perron-Mellin Formula). For all σ > σ c and all x > 0 n<x c(n) + c( x ) [[x Z]] = lim 2 T 1 2πi σ+it σ it C(s) xs s ds. where [[P]] is 1 if P is a true proposition and 0 otherwise.
9 Discrete Divide & Conquer Recurrence by Dirichlet Series For a sequence c(n) define the Dirichlet series as C(s) = n=1 c(n) n s provided it exists for R(s) > σ c for some σ c. Theorem 1 (Perron-Mellin Formula). For all σ > σ c and all x > 0 n<x c(n) + c( x ) [[x Z]] = lim 2 T 1 2πi σ+it σ it C(s) xs s ds. where [[P]] is 1 if P is a true proposition and 0 otherwise. Example: Define c(n) = T(n + 2) T(n + 1). Then T(n) = T(2) + lim T 1 2πi c+it c it T(s) (n 3 2 )s s ds for some c > σ T with T(s) = n=1 T(n + 2) T(n + 1) n s. where R(s) > σ T.
10 Assumptions Let a n be a nondecreasing sequence. Define Ã(s) = n=1 a n+2 a n+1 n s which is postulated to exists for R(s) > σ a. Example. Define a n = n σ (logn) α. Then Γ(α + 1) Γ(α + 1) Ã(s) = σ + (s σ) α+1 (s σ) + F(s), α where F(s) is analytic for R(s) > σ 1 and Γ(s) is the gamma function. Define s 0 to be the unique real root of m (b j + b j )p j s = 1. j=1 Other zeros depend on the relation among log(1/p 1 ),...,log(1/p m ).
11 Rationally and Irrationally Related Numbers Definition 1. (i) log(1/p 1 ),...,log(1/p m ) are rationally related if log(1/p 1 ),...,log(1/p m ) are integer multiples of L, that is, log(1/p j ) = n j L, n j Z, (1 j m). (ii) Otherwise log(1/p 1 ),...,log(1/p m ) are irrationally related. Example. If m = 1, then we are always in the rationally related case. For m = 2, if log(1/p 1 )/log(1/p 2 ) = m/n, (m,n integers), then rationally related. Lemma 1. (i) If log(1/p 1 ),...,log(1/p m ) are irrationally related, then s 0 is the only solution on R(s) = s 0. (ii) If log(1/p 1 ),...,log(1/p m ) are rationally related, then there are infinitely many solutions s k = s 0 + 2πik L (k Z) where log(1/p j ) are all integer multiples of L.
12 Evaluation of T(n): A Bird View We estimate T(n) by the Cauchy residue theorem. T(s) = Ã(s)+B(s) 1 m j=1 (b j +b j )ps j, T(n) = 1 2πi c+i T(s) (n 3 2 )s c i s ds
13 Main Master Theorem Theorem 2 (DISCRETE MASTER THEOREM). Let a n = Cn σa (logn) α with min{σ,α} 0. (i) If log(1/p 1 ),...,log(1/p m ) are irrationally related, then T(n) = C 1 + o(1) if σ a 0 and s 0 < 0, C 2 logn + C 2 + o(1) if σ a < s 0 = 0, C 3 (logn) α+1 (1 + +o(1)) if σ a = s 0 = 0 C 4 n s 0 (1 + o(1)) if σ a < s 0 and s 0 > 0, C 5 n s 0(logn) α+1 (1 + o(1)) if σ a = s 0 > 0 and α 1, C 5 n s 0loglogn (1 + o(1)) if σ a = s 0 > 0 and α = 1, C 6 (logn) α (1 + o(1)) if σ a = 0 and s 0 < 0, C 7 n σa (logn) α (1 + o(1)) if σ a > s 0 and σ a > 0. (ii) If log(1/p 1 ),...,log(1/p m ) are rationally related, then T(n) behaves as in the irrationally related case with the following two exceptions: T(n) = { C2 logn + Ψ 2 (logn) + o(1) if σ a < s 0 = 0, Ψ 4 (logn)n s 0 (1 + o(1)) if σ a < s 0 and s 0 > 0, where C 2 is positive and Ψ 2 (t),ψ 4 (t) are periodic functions with period L (with usually countably many discontinuities).
14 Extensions and Remarks 1. We can handle any a n sequence with Dirichlet series Ã(s): Ã(s) = g 0 (s) ( ) β0 log 1 s σa (s σ a ) α 0 + ( ) βj J log 1 s σa g j (s) (s σ a ) α j j=1 + F(s), F(s) is analytic, g 0 (σ a ) 0, β j non-negative integers, and α 0 real. Then the solutiont(n) of the divide and conquer recurrence has the form: T(n) C n σ (logn) α (loglogn) β or T(n) Ψ(logn)n s 0 σ = max{σ,s 0 }), depending whether logp 1,...logp m are irrationally or rationally related. 2. The periodic function Ψ(t) has the following building blocks λ t n 1 λ B n t logn L λ 1 +1 where λ > 1 andb n is such that n 1 B nλ (logn)/l converges absolutely. This function is discontinuous at t = {logn/l}, where{x} = x x denotes the fractional part of a real number x.
15 Examples Example 1. Irrationally Related; Case 4: T(n) = 2T( n/2 ) + 3T( n/6 ) + nlogn Here σ a = 1 since a n = nlogn. The equation 2 2 s s = 1 has the (real) solution s 0 = > 1, and finally log(1/2)/log(1/6) are irrationally related. Thus by our Master Theorem Case 4 for some constant C > 0 T(n) Cn s 0
16 Examples Example 2. Irrationally Related; Case 6: Consider the recurrence T(n) = 2T( n/2 ) + 8 n2 T( 3n/4 ) + 9 logn. Here σ a = s 0 = 2, and we deal with irrationally related case. Furthermore, Ã(s) = slog 1 s 2 + G(s) for G(s) analytic for R(s) > 1. By Master Theorem Case 6 T(n) Cn 2 loglogn. Example 3. Rationally Related (m = 1); Case 3: Next consider T(n) = T( n/2 ) + logn. Here σ a = s 0 = 0, and we have rational case (m = 1). Since Ã(s) = 1 s + G(s) we conclude T(n) C(logn) 2.
17 Examples Example 4: Karatsuba algorithm: Rationally Related (m = 1): T(n) = 3T( n/2 ) + n Here, s 0 = (log3)/(log2) = and s 0 > σ a = 1. Thus for some periodic function Ψ(t). T(n) = Ψ(logn)n log3 log2 (1 + o(1))
18 Examples Example 5. Rationally Related (m = 1). The recurrence T(n) = 1 2 T( n/2 ) + 1 n is not covered by our Master Theorem but our methodology still works. Here σ a = s 0 = 1 < 0. It follows that T(n) = C logn n + Ψ(logn) ( ) 1 + o n n for a periodic function Ψ(t). Example 6: Mergesort. Rationally Related. The mergesort recurrences are T(n) = T( n/2 ) + T( n/2 ) + n 1, Y (n) = Y( n/2 ) + Y ( n/2 ) + n/2. Here σ a = s 0 = 1 and we deal with the rationally related case. By our Master Theorem (cf. Flajolet & Golin, 1994) T(n) = Y (n) = 1 nlogn + nψ(logn) + o(n), log2 1 nlogn + nψ(logn) + o(n). 2log2
19 Boncelet s Algorithm Revisited Let a sequence X be generated by a memoryless source over alphabet A of size m with symbol probabilities p i, i A. Using the Boncelet s parsing tree, we parse X into phrases {v 1,...v n } of length l(v 1 ),..., l(v n ) with phrase probabilities P(v 1 ),...,P(v n ). Phrase Length and its Probability Generating Function: Let D n denote the phrase length and define the probability generating function as C(n,y) = E[y Dn ] It satisfies the following discrete divide and conquer recurrence: C(n,y) = y m p i C([p i n + δ i ],y) i=1 The expected phrase length d(n) = E[D n ] = C (n,1) satisfies the following discrete divide and conquer recurrence: d(n) = 1 + m p i d([p i n + δ i ]) i=1 with d(0) = = d(m 1) = 0.
20 Main Results for Boncelet s Algorithm Theorem 3. Consider anm-ary memoryless source with probabilitiesp i > 0 and the entropy rate H = m i=1 p ilog(1/p i ). (i) If log(1/p 1 ),...log(1/p m ) are irrationally related, then where d(n) = 1 H logn α H + o(1), α = E (0) H H 2 2H, H 2 = m i=1 p ilog 2 p i, and E (0) is the derivative at s = 0 of a Dirichlet series E(s) arises from the discrete nature of the recurrence. (ii) If log(1/p 1 ),...log(1/p m ) are rationally related, then d(n) = 1 H logn α + Ψ(logn) H + O(n η ) for some η > 0, where Ψ(t) is a periodic function of bounded variation that has usually an infinite number of discontinuities.
21 Redundancy of the Boncelet s Algorithm Corollary 1. Let R n denote the redundancy of the Boncelet code: R n = logn E[D n ] H = logn d(n) H. (i) If log(1/p 1 ),...log(1/p m ) are irrationally related, then R n = Hα ( ) 1 logn + o. logn (ii) If log(1/p 1 ),...log(1/p m ) are rationally related, then ( ) Hα + Ψ(logn) 1 R n = + o. logn logn Tunstall Code Redundancy: R T n = H logn ( logh H 2 2H ) ( ) 1 + o logn for irrational case; in the rational case there is aperiodic function. Example. Consider p = 1/3 and q = 2/3. Then one computes α = E (0) H H 2 2H while for the Tunstall code logh H 2 2H
22 Limiting Distribution for the Phrase length Theorem 4. Consider a memoryless source generating a sequence of length n parsed by the Boncelet algorithm. If (p 1,...,p m ) is not the uniform distribution, then the phrase length D n satisfies the central limit law, that is, D n 1 H logn (H2 ) N(0,1), logn H 3 1 H where N(0,1) denotes the standard normal distribution, and VarD n E[D n ] = logn H + O(1), H 1 ) logn 3 H ( H2 for n.
23 Sketch of Proof 1. From the recurrence we have T(s) m = Ã(s) + b j j=1 n=1 T ( p j (n + 2) + δ j ) T ( p j (n + 1) + δ j ) n s. But defining k + 2 δj n = for some integer k, we arrive at p j 2 n=1 T ( p j (n + 2) + δ j ) T ( p j (n + 1) + δ j ) n s = G j (s)+ k=1 T(k + 2) T(k + 1) ( k+2 δj ) s. p 2 j for an explicit (and simple) analytic function G j (s).
24 Sketch of Proof Continuation 2. We now compare the last sum to p s j T(s) and obtain k=1 T(k + 2) T(k + 1) ( k+2 δj ) s = p 2 j k=1 T(k + 2) T(k + 1) (k/p j ) s = p s j T(s) E j (s), where E j (s) = (T(k + 2) T(k + 1)) 1 (k/p j ) 1 ( s k+2 δj p 2 j k=1 ) s. 3. Defining E(s) = m b j E j (s) and G(s) = j=1 m b j G j (s) j=1 we finally obtain our final formula T(s) = Ã(s) + G(s) E(s) 1 m j=1 b. j p s j
25 Sketch of Proof Binary Boncelet s Algorithm 1. Let s 0 (y) be the real zero of y(p s+1 + q s+1 ) = 1, q = 1 p. 2. Define C(s,y) = n=1 which from the basic recurrence becomes C(n + 2,y) C(n + 1,y) n s. C(s,y) = (y 1) E(s,y) 1 y(p s+1 + q s+1 ), where E(s,y) converges for R(s) > s 0 (y) 1 and satisfies E(0,y) = 0 and E(s,1) = By Mellin-Perron formula and residue theorem we can prove that C(n,y) = (1 + O(y 1))n s 0 (y) (1 + o(1)) where s 0 (y) = y 1 ( H + H2 2H 1 ) (y 1) 2 + O((y 1) 3 ). 3 H
26 That s It THANK YOU
A Master Theorem for Discrete Divide and Conquer Recurrences. Dedicated to PHILIPPE FLAJOLET
A Master Theorem for Discrete Divide and Conquer Recurrences Wojciech Szpankowski Department of Computer Science Purdue University U.S.A. September 6, 2012 Uniwersytet Jagieloński, Kraków, 2012 Dedicated
More informationA Master Theorem for Discrete Divide and Conquer Recurrences
A Master Theorem for Discrete Divide and Conquer Recurrences MICHAEL DRMOTA and WOJCIECH SZPANKOWSKI TU Wien and Purdue University Divide-and-conquer recurrences are one of the most studied equations in
More informationVariable-to-Variable Codes with Small Redundancy Rates
Variable-to-Variable Codes with Small Redundancy Rates M. Drmota W. Szpankowski September 25, 2004 This research is supported by NSF, NSA and NIH. Institut f. Diskrete Mathematik und Geometrie, TU Wien,
More informationMinimax Redundancy for Large Alphabets by Analytic Methods
Minimax Redundancy for Large Alphabets by Analytic Methods Wojciech Szpankowski Purdue University W. Lafayette, IN 47907 March 15, 2012 NSF STC Center for Science of Information CISS, Princeton 2012 Joint
More informationDigital Trees and Memoryless Sources: from Arithmetics to Analysis
Digital Trees and Memoryless Sources: from Arithmetics to Analysis Philippe Flajolet, Mathieu Roux, Brigitte Vallée AofA 2010, Wien 1 What is a digital tree, aka TRIE? = a data structure for dynamic dictionaries
More informationA Master Theorem for Discrete Divide and Conquer Recurrences
A Master Theorem for Discrete Divide and Conquer Recurrences Michael Drmota Wojciech Szpankowski Dedicated to Philippe Flajolet 948- Abstract Divide-and-conquer recurrences are one of the most studied
More informationString Complexity. Dedicated to Svante Janson for his 60 Birthday
String Complexity Wojciech Szpankowski Purdue University W. Lafayette, IN 47907 June 1, 2015 Dedicated to Svante Janson for his 60 Birthday Outline 1. Working with Svante 2. String Complexity 3. Joint
More informationMultiple Choice Tries and Distributed Hash Tables
Multiple Choice Tries and Distributed Hash Tables Luc Devroye and Gabor Lugosi and Gahyun Park and W. Szpankowski January 3, 2007 McGill University, Montreal, Canada U. Pompeu Fabra, Barcelona, Spain U.
More informationAnalytic Information Theory: From Shannon to Knuth and Back. Knuth80: Piteaa, Sweden, 2018 Dedicated to Don E. Knuth
Analytic Information Theory: From Shannon to Knuth and Back Wojciech Szpankowski Center for Science of Information Purdue University January 7, 2018 Knuth80: Piteaa, Sweden, 2018 Dedicated to Don E. Knuth
More informationSIGNAL COMPRESSION Lecture 7. Variable to Fix Encoding
SIGNAL COMPRESSION Lecture 7 Variable to Fix Encoding 1. Tunstall codes 2. Petry codes 3. Generalized Tunstall codes for Markov sources (a presentation of the paper by I. Tabus, G. Korodi, J. Rissanen.
More informationProbabilistic analysis of the asymmetric digital search trees
Int. J. Nonlinear Anal. Appl. 6 2015 No. 2, 161-173 ISSN: 2008-6822 electronic http://dx.doi.org/10.22075/ijnaa.2015.266 Probabilistic analysis of the asymmetric digital search trees R. Kazemi a,, M. Q.
More informationA One-to-One Code and Its Anti-Redundancy
A One-to-One Code and Its Anti-Redundancy W. Szpankowski Department of Computer Science, Purdue University July 4, 2005 This research is supported by NSF, NSA and NIH. Outline of the Talk. Prefix Codes
More informationAnalytic Pattern Matching: From DNA to Twitter. AxA Workshop, Venice, 2016 Dedicated to Alberto Apostolico
Analytic Pattern Matching: From DNA to Twitter Wojciech Szpankowski Purdue University W. Lafayette, IN 47907 June 19, 2016 AxA Workshop, Venice, 2016 Dedicated to Alberto Apostolico Joint work with Philippe
More informationTunstall Code, Khodak Variations, and Random Walks
Tunstall Code, Khodak Variations, and Random Walks April 15, 2010 Michael Drmota Yuriy A. Reznik Wojciech Szpankowski Inst. Discrete Math. and Geometry Qualcomm Inc. Dept. Computer Science TU Wien 5775
More informationChapter 1 Divide and Conquer Polynomial Multiplication Algorithm Theory WS 2015/16 Fabian Kuhn
Chapter 1 Divide and Conquer Polynomial Multiplication Algorithm Theory WS 2015/16 Fabian Kuhn Formulation of the D&C principle Divide-and-conquer method for solving a problem instance of size n: 1. Divide
More informationChapter 1 Divide and Conquer Algorithm Theory WS 2016/17 Fabian Kuhn
Chapter 1 Divide and Conquer Algorithm Theory WS 2016/17 Fabian Kuhn Formulation of the D&C principle Divide-and-conquer method for solving a problem instance of size n: 1. Divide n c: Solve the problem
More informationDivide and Conquer Algorithms
Divide and Conquer Algorithms T. M. Murali February 19, 2013 Divide and Conquer Break up a problem into several parts. Solve each part recursively. Solve base cases by brute force. Efficiently combine
More informationDivide and Conquer. Andreas Klappenecker
Divide and Conquer Andreas Klappenecker The Divide and Conquer Paradigm The divide and conquer paradigm is important general technique for designing algorithms. In general, it follows the steps: - divide
More informationDivide and Conquer Algorithms
Divide and Conquer Algorithms T. M. Murali March 17, 2014 Divide and Conquer Break up a problem into several parts. Solve each part recursively. Solve base cases by brute force. Efficiently combine solutions
More informationChapter 2: Source coding
Chapter 2: meghdadi@ensil.unilim.fr University of Limoges Chapter 2: Entropy of Markov Source Chapter 2: Entropy of Markov Source Markov model for information sources Given the present, the future is independent
More informationCoding on Countably Infinite Alphabets
Coding on Countably Infinite Alphabets Non-parametric Information Theory Licence de droits d usage Outline Lossless Coding on infinite alphabets Source Coding Universal Coding Infinite Alphabets Enveloppe
More informationCSE 421 Algorithms. T(n) = at(n/b) + n c. Closest Pair Problem. Divide and Conquer Algorithms. What you really need to know about recurrences
CSE 421 Algorithms Richard Anderson Lecture 13 Divide and Conquer What you really need to know about recurrences Work per level changes geometrically with the level Geometrically increasing (x > 1) The
More informationOn universal types. Gadiel Seroussi Information Theory Research HP Laboratories Palo Alto HPL September 6, 2004*
On universal types Gadiel Seroussi Information Theory Research HP Laboratories Palo Alto HPL-2004-153 September 6, 2004* E-mail: gadiel.seroussi@hp.com method of types, type classes, Lempel-Ziv coding,
More informationRecurrence Relations
Recurrence Relations Analysis Tools S.V. N. (vishy) Vishwanathan University of California, Santa Cruz vishy@ucsc.edu January 15, 2016 S.V. N. Vishwanathan (UCSC) CMPS101 1 / 29 Recurrences Outline 1 Recurrences
More informationRENEWAL THEORY IN ANALYSIS OF TRIES AND STRINGS: EXTENDED ABSTRACT
RENEWAL THEORY IN ANALYSIS OF TRIES AND STRINGS: EXTENDED ABSTRACT SVANTE JANSON Abstract. We give a survey of a number of simple applications of renewal theory to problems on random strings, in particular
More informationSource Coding. Master Universitario en Ingeniería de Telecomunicación. I. Santamaría Universidad de Cantabria
Source Coding Master Universitario en Ingeniería de Telecomunicación I. Santamaría Universidad de Cantabria Contents Introduction Asymptotic Equipartition Property Optimal Codes (Huffman Coding) Universal
More informationAverage Redundancy for Known Sources: Ubiquitous Trees in Source Coding
Discrete Mathematics and Theoretical Computer Science DMTCS vol. (subm.), by the authors, 1 1 Average Redundancy for Known Sources: Ubiquitous Trees in Source Coding Wojciech Szpanowsi Department of Computer
More informationExistence of a Limit on a Dense Set, and. Construction of Continuous Functions on Special Sets
Existence of a Limit on a Dense Set, and Construction of Continuous Functions on Special Sets REU 2012 Recap: Definitions Definition Given a real-valued function f, the limit of f exists at a point c R
More informationDivide and Conquer. Recurrence Relations
Divide and Conquer Recurrence Relations Divide-and-Conquer Strategy: Break up problem into parts. Solve each part recursively. Combine solutions to sub-problems into overall solution. 2 MergeSort Mergesort.
More informationOn Universal Types. Gadiel Seroussi Hewlett-Packard Laboratories Palo Alto, California, USA. University of Minnesota, September 14, 2004
On Universal Types Gadiel Seroussi Hewlett-Packard Laboratories Palo Alto, California, USA University of Minnesota, September 14, 2004 Types for Parametric Probability Distributions A = finite alphabet,
More informationOn the Average Path Length of Complete m-ary Trees
1 2 3 47 6 23 11 Journal of Integer Sequences, Vol. 17 2014, Article 14.6.3 On the Average Path Length of Complete m-ary Trees M. A. Nyblom School of Mathematics and Geospatial Science RMIT University
More informationData structures Exercise 1 solution. Question 1. Let s start by writing all the functions in big O notation:
Data structures Exercise 1 solution Question 1 Let s start by writing all the functions in big O notation: f 1 (n) = 2017 = O(1), f 2 (n) = 2 log 2 n = O(n 2 ), f 3 (n) = 2 n = O(2 n ), f 4 (n) = 1 = O
More informationCPS 616 DIVIDE-AND-CONQUER 6-1
CPS 616 DIVIDE-AND-CONQUER 6-1 DIVIDE-AND-CONQUER Approach 1. Divide instance of problem into two or more smaller instances 2. Solve smaller instances recursively 3. Obtain solution to original (larger)
More informationNotes for Lecture 3... x 4
Stanford University CS254: Computational Complexity Notes 3 Luca Trevisan January 14, 2014 Notes for Lecture 3 In this lecture we introduce the computational model of boolean circuits and prove that polynomial
More informationLecture 4 : Adaptive source coding algorithms
Lecture 4 : Adaptive source coding algorithms February 2, 28 Information Theory Outline 1. Motivation ; 2. adaptive Huffman encoding ; 3. Gallager and Knuth s method ; 4. Dictionary methods : Lempel-Ziv
More informationarxiv: v22 [math.gm] 20 Sep 2018
O RIEMA HYPOTHESIS arxiv:96.464v [math.gm] Sep 8 RUIMIG ZHAG Abstract. In this work we present a proof to the celebrated Riemann hypothesis. In it we first apply the Plancherel theorem properties of the
More information#A5 INTEGERS 18A (2018) EXPLICIT EXAMPLES OF p-adic NUMBERS WITH PRESCRIBED IRRATIONALITY EXPONENT
#A5 INTEGERS 8A (208) EXPLICIT EXAMPLES OF p-adic NUMBERS WITH PRESCRIBED IRRATIONALITY EXPONENT Yann Bugeaud IRMA, UMR 750, Université de Strasbourg et CNRS, Strasbourg, France bugeaud@math.unistra.fr
More information7 Asymptotics for Meromorphic Functions
Lecture G jacques@ucsd.edu 7 Asymptotics for Meromorphic Functions Hadamard s Theorem gives a broad description of the exponential growth of coefficients in power series, but the notion of exponential
More informationMULTI-RESTRAINED STIRLING NUMBERS
MULTI-RESTRAINED STIRLING NUMBERS JI YOUNG CHOI DEPARTMENT OF MATHEMATICS SHIPPENSBURG UNIVERSITY SHIPPENSBURG, PA 17257, U.S.A. Abstract. Given positive integers n, k, and m, the (n, k)-th m- restrained
More informationCounting Markov Types
Counting Markov Types Philippe Jacquet, Charles Knessl, Wojciech Szpankowski To cite this version: Philippe Jacquet, Charles Knessl, Wojciech Szpankowski. Counting Markov Types. Drmota, Michael and Gittenberger,
More informationTHE REAL NUMBERS Chapter #4
FOUNDATIONS OF ANALYSIS FALL 2008 TRUE/FALSE QUESTIONS THE REAL NUMBERS Chapter #4 (1) Every element in a field has a multiplicative inverse. (2) In a field the additive inverse of 1 is 0. (3) In a field
More informationNotes for Recitation 14
6.04/18.06J Mathematics for Computer Science October 4, 006 Tom Leighton and Marten van Dijk Notes for Recitation 14 1 The Akra-Bazzi Theorem Theorem 1 (Akra-Bazzi, strong form). Suppose that: is defined
More informationCSE101: Design and Analysis of Algorithms. Ragesh Jaiswal, CSE, UCSD
Greedy s Greedy s Shortest path Claim 2: Let S be a subset of vertices containing s such that we know the shortest path length l(s, u) from s to any vertex in u S. Let e = (u, v) be an edge such that 1
More informationIn 1826, Abel proved the following result for real power series. Let f(x) =
Chapter 6 Tauberian theorems 6. Introduction In 826, Abel proved the following result for real power series. Let f(x) = n=0 a nx n be a power series with coefficients a n R that converges on the real interval
More informationDesign and Analysis of Algorithms
CSE 101, Winter 2018 Design and Analysis of Algorithms Lecture 4: Divide and Conquer (I) Class URL: http://vlsicad.ucsd.edu/courses/cse101-w18/ Divide and Conquer ( DQ ) First paradigm or framework DQ(S)
More informationData Compression Techniques
Data Compression Techniques Part 1: Entropy Coding Lecture 4: Asymmetric Numeral Systems Juha Kärkkäinen 08.11.2017 1 / 19 Asymmetric Numeral Systems Asymmetric numeral systems (ANS) is a recent entropy
More informationSearch Algorithms. Analysis of Algorithms. John Reif, Ph.D. Prepared by
Search Algorithms Analysis of Algorithms Prepared by John Reif, Ph.D. Search Algorithms a) Binary Search: average case b) Interpolation Search c) Unbounded Search (Advanced material) Readings Reading Selection:
More informationCS 5321: Advanced Algorithms - Recurrence. Acknowledgement. Outline. Ali Ebnenasir Department of Computer Science Michigan Technological University
CS 5321: Advanced Algorithms - Recurrence Ali Ebnenasir Department of Computer Science Michigan Technological University Acknowledgement Eric Torng Moon Jung Chung Charles Ofria Outline Motivating example:
More informationMOMENTS OF HYPERGEOMETRIC HURWITZ ZETA FUNCTIONS
MOMENTS OF HYPERGEOMETRIC HURWITZ ZETA FUNCTIONS ABDUL HASSEN AND HIEU D. NGUYEN Abstract. This paper investigates a generalization the classical Hurwitz zeta function. It is shown that many of the properties
More informationApplications. More Counting Problems. Complexity of Algorithms
Recurrences Applications More Counting Problems Complexity of Algorithms Part I Recurrences and Binomial Coefficients Paths in a Triangle P(0, 0) P(1, 0) P(1,1) P(2, 0) P(2,1) P(2, 2) P(3, 0) P(3,1) P(3,
More informationMidterm 2 for CS 170
UC Berkeley CS 170 Midterm 2 Lecturer: Gene Myers November 9 Midterm 2 for CS 170 Print your name:, (last) (first) Sign your name: Write your section number (e.g. 101): Write your sid: One page of notes
More informationRecurrence Relations
Recurrence Relations Winter 2017 Recurrence Relations Recurrence Relations A recurrence relation for the sequence {a n } is an equation that expresses a n in terms of one or more of the previous terms
More informationJumping Sequences. Steve Butler 1. (joint work with Ron Graham and Nan Zang) University of California Los Angelese
Jumping Sequences Steve Butler 1 (joint work with Ron Graham and Nan Zang) 1 Department of Mathematics University of California Los Angelese www.math.ucla.edu/~butler UCSD Combinatorics Seminar 14 October
More informationCS 5321: Advanced Algorithms Analysis Using Recurrence. Acknowledgement. Outline
CS 5321: Advanced Algorithms Analysis Using Recurrence Ali Ebnenasir Department of Computer Science Michigan Technological University Acknowledgement Eric Torng Moon Jung Chung Charles Ofria Outline Motivating
More information3F1 Information Theory, Lecture 3
3F1 Information Theory, Lecture 3 Jossy Sayir Department of Engineering Michaelmas 2013, 29 November 2013 Memoryless Sources Arithmetic Coding Sources with Memory Markov Example 2 / 21 Encoding the output
More informationMA008/MIIZ01 Design and Analysis of Algorithms Lecture Notes 3
MA008 p.1/37 MA008/MIIZ01 Design and Analysis of Algorithms Lecture Notes 3 Dr. Markus Hagenbuchner markus@uow.edu.au. MA008 p.2/37 Exercise 1 (from LN 2) Asymptotic Notation When constants appear in exponents
More informationStochastic Processes (Week 6)
Stochastic Processes (Week 6) October 30th, 2014 1 Discrete-time Finite Markov Chains 2 Countable Markov Chains 3 Continuous-Time Markov Chains 3.1 Poisson Process 3.2 Finite State Space 3.2.1 Kolmogrov
More informationEE5139R: Problem Set 4 Assigned: 31/08/16, Due: 07/09/16
EE539R: Problem Set 4 Assigned: 3/08/6, Due: 07/09/6. Cover and Thomas: Problem 3.5 Sets defined by probabilities: Define the set C n (t = {x n : P X n(x n 2 nt } (a We have = P X n(x n P X n(x n 2 nt
More informationLecture 4: Exponential family of distributions and generalized linear model (GLM) (Draft: version 0.9.2)
Lectures on Machine Learning (Fall 2017) Hyeong In Choi Seoul National University Lecture 4: Exponential family of distributions and generalized linear model (GLM) (Draft: version 0.9.2) Topics to be covered:
More informationAlgorithms, Design and Analysis. Order of growth. Table 2.1. Big-oh. Asymptotic growth rate. Types of formulas for basic operation count
Types of formulas for basic operation count Exact formula e.g., C(n) = n(n-1)/2 Algorithms, Design and Analysis Big-Oh analysis, Brute Force, Divide and conquer intro Formula indicating order of growth
More informationDigital search trees JASS
Digital search trees Analysis of different digital trees with Rice s integrals. JASS Nicolai v. Hoyningen-Huene 28.3.2004 28.3.2004 JASS 04 - Digital search trees 1 content Tree Digital search tree: Definition
More informationData Structures in Java
Data Structures in Java Lecture 20: Algorithm Design Techniques 12/2/2015 Daniel Bauer 1 Algorithms and Problem Solving Purpose of algorithms: find solutions to problems. Data Structures provide ways of
More informationDivide and Conquer. CSE21 Winter 2017, Day 9 (B00), Day 6 (A00) January 30,
Divide and Conquer CSE21 Winter 2017, Day 9 (B00), Day 6 (A00) January 30, 2017 http://vlsicad.ucsd.edu/courses/cse21-w17 Merging sorted lists: WHAT Given two sorted lists a 1 a 2 a 3 a k b 1 b 2 b 3 b
More informationDivide and Conquer. Andreas Klappenecker. [based on slides by Prof. Welch]
Divide and Conquer Andreas Klappenecker [based on slides by Prof. Welch] Divide and Conquer Paradigm An important general technique for designing algorithms: divide problem into subproblems recursively
More informationDivide and Conquer: Polynomial Multiplication Version of October 1 / 7, 24201
Divide and Conquer: Polynomial Multiplication Version of October 7, 2014 Divide and Conquer: Polynomial Multiplication Version of October 1 / 7, 24201 Outline Outline: Introduction The polynomial multiplication
More informationChapter 2 Date Compression: Source Coding. 2.1 An Introduction to Source Coding 2.2 Optimal Source Codes 2.3 Huffman Code
Chapter 2 Date Compression: Source Coding 2.1 An Introduction to Source Coding 2.2 Optimal Source Codes 2.3 Huffman Code 2.1 An Introduction to Source Coding Source coding can be seen as an efficient way
More informationRegularity of the density for the stochastic heat equation
Regularity of the density for the stochastic heat equation Carl Mueller 1 Department of Mathematics University of Rochester Rochester, NY 15627 USA email: cmlr@math.rochester.edu David Nualart 2 Department
More informationCoding for Discrete Source
EGR 544 Communication Theory 3. Coding for Discrete Sources Z. Aliyazicioglu Electrical and Computer Engineering Department Cal Poly Pomona Coding for Discrete Source Coding Represent source data effectively
More informationDivide and Conquer. Maximum/minimum. Median finding. CS125 Lecture 4 Fall 2016
CS125 Lecture 4 Fall 2016 Divide and Conquer We have seen one general paradigm for finding algorithms: the greedy approach. We now consider another general paradigm, known as divide and conquer. We have
More informationDesign and Analysis of Algorithms
CSE 101, Winter 2018 Design and Analysis of Algorithms Lecture 5: Divide and Conquer (Part 2) Class URL: http://vlsicad.ucsd.edu/courses/cse101-w18/ A Lower Bound on Convex Hull Lecture 4 Task: sort the
More informationAnalytic Information Theory and Beyond
Analytic Information Theory and Beyond W. Szpankowski Department of Computer Science Purdue University W. Lafayette, IN 47907 May 12, 2010 AofA and IT logos Frankfurt 2010 Research supported by NSF Science
More informationInformation Theory with Applications, Math6397 Lecture Notes from September 30, 2014 taken by Ilknur Telkes
Information Theory with Applications, Math6397 Lecture Notes from September 3, 24 taken by Ilknur Telkes Last Time Kraft inequality (sep.or) prefix code Shannon Fano code Bound for average code-word length
More informationMarkov Chains, Stochastic Processes, and Matrix Decompositions
Markov Chains, Stochastic Processes, and Matrix Decompositions 5 May 2014 Outline 1 Markov Chains Outline 1 Markov Chains 2 Introduction Perron-Frobenius Matrix Decompositions and Markov Chains Spectral
More informationChapter 5: Data Compression
Chapter 5: Data Compression Definition. A source code C for a random variable X is a mapping from the range of X to the set of finite length strings of symbols from a D-ary alphabet. ˆX: source alphabet,
More informationA design paradigm. Divide and conquer: (When) does decomposing a problem into smaller parts help? 09/09/ EECS 3101
A design paradigm Divide and conquer: (When) does decomposing a problem into smaller parts help? 09/09/17 112 Multiplying complex numbers (from Jeff Edmonds slides) INPUT: Two pairs of integers, (a,b),
More information7.11 A proof involving composition Variation in terminology... 88
Contents Preface xi 1 Math review 1 1.1 Some sets............................. 1 1.2 Pairs of reals........................... 3 1.3 Exponentials and logs...................... 4 1.4 Some handy functions......................
More informationEECS 229A Spring 2007 * * (a) By stationarity and the chain rule for entropy, we have
EECS 229A Spring 2007 * * Solutions to Homework 3 1. Problem 4.11 on pg. 93 of the text. Stationary processes (a) By stationarity and the chain rule for entropy, we have H(X 0 ) + H(X n X 0 ) = H(X 0,
More informationPart II. Number Theory. Year
Part II Year 2017 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 2017 Paper 3, Section I 1G 70 Explain what is meant by an Euler pseudoprime and a strong pseudoprime. Show that 65 is an Euler
More informationChapter 3 Source Coding. 3.1 An Introduction to Source Coding 3.2 Optimal Source Codes 3.3 Shannon-Fano Code 3.4 Huffman Code
Chapter 3 Source Coding 3. An Introduction to Source Coding 3.2 Optimal Source Codes 3.3 Shannon-Fano Code 3.4 Huffman Code 3. An Introduction to Source Coding Entropy (in bits per symbol) implies in average
More informationLecture 3: Probability Measures - 2
Lecture 3: Probability Measures - 2 1. Continuation of measures 1.1 Problem of continuation of a probability measure 1.2 Outer measure 1.3 Lebesgue outer measure 1.4 Lebesgue continuation of an elementary
More informationDivide and Conquer. Arash Rafiey. 27 October, 2016
27 October, 2016 Divide the problem into a number of subproblems Divide the problem into a number of subproblems Conquer the subproblems by solving them recursively or if they are small, there must be
More information3F1 Information Theory, Lecture 3
3F1 Information Theory, Lecture 3 Jossy Sayir Department of Engineering Michaelmas 2011, 28 November 2011 Memoryless Sources Arithmetic Coding Sources with Memory 2 / 19 Summary of last lecture Prefix-free
More informationAn Operator Theoretical Approach to Nonlocal Differential Equations
An Operator Theoretical Approach to Nonlocal Differential Equations Joshua Lee Padgett Department of Mathematics and Statistics Texas Tech University Analysis Seminar November 27, 2017 Joshua Lee Padgett
More informationThe Moments of the Profile in Random Binary Digital Trees
Journal of mathematics and computer science 6(2013)176-190 The Moments of the Profile in Random Binary Digital Trees Ramin Kazemi and Saeid Delavar Department of Statistics, Imam Khomeini International
More informationNotes for Lecture 3... x 4
Stanford University CS254: Computational Complexity Notes 3 Luca Trevisan January 18, 2012 Notes for Lecture 3 In this lecture we introduce the computational model of boolean circuits and prove that polynomial
More informationCUTS OF LINEAR ORDERS
CUTS OF LINEAR ORDERS ASHER M. KACH AND ANTONIO MONTALBÁN Abstract. We study the connection between the number of ascending and descending cuts of a linear order, its classical size, and its effective
More informationarxiv: v6 [math.nt] 12 Sep 2017
Counterexamples to the conjectured transcendence of /α) k, its closed-form summation and extensions to polygamma functions and zeta series arxiv:09.44v6 [math.nt] Sep 07 F. M. S. Lima Institute of Physics,
More informationOutline. Part 5. Computa0onal Complexity (3) Compound Interest 2/15/12. Recurrence Rela0ons: An Overview. Recurrence Rela0ons: Formal Defini0on
Outline Part 5. Computa0onal Complexity (3) Recurrence Relations Divide and Conquer Understanding the Complexity of Algorithms CS 200 Algorithms and Data Structures 1 2 Recurrence Rela0ons: An Overview
More informationSIGNAL COMPRESSION Lecture Shannon-Fano-Elias Codes and Arithmetic Coding
SIGNAL COMPRESSION Lecture 3 4.9.2007 Shannon-Fano-Elias Codes and Arithmetic Coding 1 Shannon-Fano-Elias Coding We discuss how to encode the symbols {a 1, a 2,..., a m }, knowing their probabilities,
More informationCS173 Running Time and Big-O. Tandy Warnow
CS173 Running Time and Big-O Tandy Warnow CS 173 Running Times and Big-O analysis Tandy Warnow Today s material We will cover: Running time analysis Review of running time analysis of Bubblesort Review
More informationPROBABILISTIC BEHAVIOR OF ASYMMETRIC LEVEL COMPRESSED TRIES
PROBABILISTIC BEAVIOR OF ASYMMETRIC LEVEL COMPRESSED TRIES Luc Devroye Wojcieh Szpankowski School of Computer Science Department of Computer Sciences McGill University Purdue University 3450 University
More informationAsymptotic and Exact Poissonized Variance in the Analysis of Random Digital Trees (joint with Hsien-Kuei Hwang and Vytas Zacharovas)
Asymptotic and Exact Poissonized Variance in the Analysis of Random Digital Trees (joint with Hsien-Kuei Hwang and Vytas Zacharovas) Michael Fuchs Institute of Applied Mathematics National Chiao Tung University
More informationECE 587 / STA 563: Lecture 5 Lossless Compression
ECE 587 / STA 563: Lecture 5 Lossless Compression Information Theory Duke University, Fall 2017 Author: Galen Reeves Last Modified: October 18, 2017 Outline of lecture: 5.1 Introduction to Lossless Source
More informationChapter Four Gelfond s Solution of Hilbert s Seventh Problem (Revised January 2, 2011)
Chapter Four Gelfond s Solution of Hilbert s Seventh Problem (Revised January 2, 2011) Before we consider Gelfond s, and then Schneider s, complete solutions to Hilbert s seventh problem let s look back
More informationChapter 2. Recurrence Relations. Divide and Conquer. Divide and Conquer Strategy. Another Example: Merge Sort. Merge Sort Example. Merge Sort Example
Recurrence Relations Chapter 2 Divide and Conquer Equation or an inequality that describes a function by its values on smaller inputs. Recurrence relations arise when we analyze the running time of iterative
More informationECE 587 / STA 563: Lecture 5 Lossless Compression
ECE 587 / STA 563: Lecture 5 Lossless Compression Information Theory Duke University, Fall 28 Author: Galen Reeves Last Modified: September 27, 28 Outline of lecture: 5. Introduction to Lossless Source
More informationKolmogorov complexity
Kolmogorov complexity In this section we study how we can define the amount of information in a bitstring. Consider the following strings: 00000000000000000000000000000000000 0000000000000000000000000000000000000000
More informationDivide-and-Conquer Algorithms and Recurrence Relations. Niloufar Shafiei
Divide-and-Conquer Algorithms and Recurrence Relations Niloufar Shafiei Divide-and-conquer algorithms Divide-and-conquer algorithms: 1. Dividing the problem into smaller sub-problems 2. Solving those sub-problems
More informationA Note on a Problem Posed by D. E. Knuth on a Satisfiability Recurrence
A Note on a Problem Posed by D. E. Knuth on a Satisfiability Recurrence Dedicated to Philippe Flajolet 1948-2011 July 4, 2013 Philippe Jacquet Charles Knessl 1 Wojciech Szpankowski 2 Bell Labs Dept. Math.
More informationFrom the Discrete to the Continuous, and Back... Philippe Flajolet INRIA, France
From the Discrete to the Continuous, and Back... Philippe Flajolet INRIA, France 1 Discrete structures in... Combinatorial mathematics Computer science: data structures & algorithms Information and communication
More information