CHAPTER 17 Amortized Analysis
|
|
- Christiana Ginger Hudson
- 5 years ago
- Views:
Transcription
1 CHAPTER 7 Amortzed Analyss In an amortzed analyss, the tme requred to perform a sequence of data structure operatons s averaged over all the operatons performed. It can be used to show that the average cost of an operaton s small, f one averages over a sequence of operatons. Whle a partcular operaton n sequence may be expensve, ths operaton may not occur often enough to make the average cost expensve. An amortzed analyss guarantees the average performance of each operaton n the worst case. It determnes the average tme wthout the use of probablty. Three methods are covered n text. The man dfference s the way the cost s assgned. Aggregate Method Characterstcs Computes the worst case tme T(n) for a sequence of n operatons. The amortzed cost (the average cost) per operaton s T(n)/n Gves average performance of each operaton n the worst case. Ths method s less precse than other methods, as all operatons are assgned the same cost. Advanced Algorthms, Feodor F. Dragan, Kent State Unversty
2 Aggregate Method An Aggregate Method Example: (Stack Operatons) Assume the followng three operatons on a stack: push(s,x) - pushes x onto stack S pop(s) - pops & returns top of stack S multpop(s,k) - pops and returns the top mn{k, S } tems of S. Worst case cost for Multpop s O(n) n successve calls to Multpop would cost O(n ). Consder a sequence of n push, pop and multpop operatons on an ntally empty stack. Ths O(n ) cost s unfar. Each tem can be popped only once for each tme t s pushed. In a sequence of n mxed operatons, the most tmes multpop can be called s n/. Snce the cost of push and pop s O(), the cost of n stack operatons s O(n). Therefore, the average cost of each stack operaton n ths sequence s O(n)/n or O(). Advanced Algorthms, Feodor F. Dragan, Kent State Unversty
3 Aggregate Method (a bnary counter) We use an array A[0,,,k-] of bts, where length(a)=k, as a counter. A bnary number x that s stored n the counter has ts lowest order bt n A[0] and k hghest-order bt n A[k-], so that x = A[ ]. Intally x=0 (A[I]=0 for all ). = 0 To add (modulo k ) to the value of the counter we use the followng procedure. INCREMENT(A) { =0 whle <length[a] and A[]= do { A[]=0; =+ } f <length[a] then A[]= } A sngle executon of INCREMENT takes O(k) n the worst case (f A contans all s). A sequence of n INCREMENT operatons on an ntally zero counter takes tme O(kn) n the worst case. (??? Is ths true?) We can tghten our analyss to yeld a worst case cost of O(n) operatons for a sequence of n INCREMENT s by observng that not all bts flp each tme INCREMENT s called. For =0,, [log n], bt A[] flps n / tmes n a sequence of n INCREMENT operatons on an ntally zero counter. For >[log n] bt A[] never flps at all. The total number of flps n the sequence s thus logn n / < n = n. = 0 = 0 Therefore, the worst-case tme for a sequence of n INCREMENT operatons on an ntal zero counter s O(n). The average cost of each operaton, and therefore the amortzed cost per operaton, s O(n)/n=O(). Advanced Algorthms, Feodor F. Dragan, Kent State Unversty 3
4 Accountng Method The Accountng Method Characterstcs Assgn dfferent (artfcal) charges to dfferent operatons. The amount we charge an operaton s called ts amortzed cost. When an operaton s amortzed cost exceeds ts actual cost, the dfference s assgned to specfc objects n the data structure as credt. Credt can be used later on to help pay for operatons whose amortzed cost s less than ther actual cost. The balance n the bank account s not allowed to become negatve. The sum of the amortzed costs for any sequence of operatons must be an upper bound for the actual total cost of these operatons. The amortzed cost of each operaton must be chosen wsely n order to pay for each operaton on or before the cost s ncurred. An Accountng Method Example: (stack operatons) Recall the actual costs of these operatons were push (S,x) pop (S) multpop(s,k) mn(k, S ) (complexty depends on k) The amortzed costs assgned are push pop 0 multpop 0 Observe that the amortzed cost of each operaton s O(). Advanced Algorthms, Feodor F. Dragan, Kent State Unversty 4
5 Accountng Method (examples) We must now show that we can pay for any sequence of stack operatons by chargng the amortzed cost (recall that we start wth ntally empty stack). The two unt costs assocated wth each push s used as follows: unt s used to pay the cost of the push. unt s collected n advance to pay for a potental future pop. For any sequence of n operatons of push, pop, and multpop, the total amortzed cost s an upper bound on the total actual cost. Snce the total amortzed cost s O(n), so s the total actual cost. In ncrementng a bnary counter, we observed earler, the runnng tme of ths operaton s proportonal to the number of bts flpped, whch we wll use as our cost for ths example. For the amortzed analyss, let as charge an amortzed cost of dollars to set a bt to. When a bt s set, we use dollar to pay for the actual settng of the bt, and we place the other dollar on the bt as credt to be used later when we flp the bt back to 0. The amortzed cost of an INCREMENT operaton s at most dollars INCREMENT(A) { =0 whle <length[a] and A[]= do { A[]=0; =+ } f <length[a] then A[]= } Thus, for n INCREMENT operatons, the total amortzed cost s O(n), whch bounds the total actual cost. Advanced Algorthms, Feodor F. Dragan, Kent State Unversty 5
6 Ths method stores prepayments as a potental to pay for future operatons. The potental stored s assocated wth the entre data structure rather than wth a specfc tem n that data structure. Notaton: D o s the ntal data structure (e.g., stack) D s the data structure after the th operaton c s the actual cost of the th operaton. The potental functon Φ (.e., ps) maps each D to ts potental value, Φ(D ). The amortzed cost ĉ of the th operaton s defned by ĉ = c + Φ(D ) - Φ(D - ). Note: ĉ = (actual cost) + (change n potental) The total amortzed cost s n = ĉ The Potental Method n = [ c + Φ ( D ) Φ ( D )] = = n ( c ) + Φ ( D n ) Φ ( D 0 = By requrng that Φ( D ) Φ( D0 ) for all, we nsure that the total amortzed cost s an upper bound for the actual cost for any sequence (Îchoose approprate Φ). ) Advanced Algorthms, Feodor F. Dragan, Kent State Unversty 6
7 The Potental Method (cont.) If Φ( D ) Φ( D ) > 0, then ĉ s an overcharge for the th operaton and causes an ncrease n potental. Smlarly, f Φ( D ) Φ( D ) < 0, then ĉ s an undercharge and results n a decrease n potental. A Potental Method Example: (Stack Operatons) The data structure D s ntally an empty stack. Let Φ(D) be the number of tems n the stack. Then Φ(D 0 ) = 0 and Φ(D ) 0 push operaton : If the th operaton on D s a push, then Φ(D ) - Φ(D - ) = Φ(D - ) + - Φ(D - ) = The amortzed cost for a push s ĉ = c + [Φ(D ) - Φ(D - )] = + = multpop operaton: If the th operaton s multpop(d,k) and k = mn{ D, k}, then c = k and ĉ = c + [Φ(D ) - Φ(D - )] = k - k = 0 pop operaton: Hence, f pop s the th operaton, then ĉ = 0 COST ANALYSIS: The amortzed cost for each operaton s O(). The amortzed cost of n operatons s O(n). The upper bound for the total cost s O(n). Advanced Algorthms, Feodor F. Dragan, Kent State Unversty 7
8 Dynamc Table Problem Problem: Consder the cost of a sequence of TABLE-DELETE and TABLE- INSERT commands for a dynamc table Normally, cost of a nserton or deleton s. However cost s large f a table expanson or contracton s trggered by the add or delete. Analyss gven here s ndependent of the data structure used. Restrcted Dynamc Table Problem (Insertons) Only nsertons are allowed. Goal: Try to keep the table as small as possble. Must enlarge table when too many tems nserted Proposed Idea for Algorthm:. Intalze table sze to m =. Insert elements untl the number n of elements satsfes n > m. 3. Generate a new table of sze m and set m m 4. Re-nsert old elements nto the new table. 5. Go to step. Let c be the cost of the th nsert. Then = otherwse s an exact power of Advanced Algorthms, Feodor F. Dragan, Kent State Unversty 8 c f -
9 Aggregate Analyss Worst case s O(n). If repeated n tmes, s a worst case of O(n ) possble? Illustraton: Inserton Sze Cost Aggregate Analyss: n n nserts cost lgn ln n + j = c n + j= 0 = n + n + ( n ) < 3 n. Average cost of each operaton = (Total Cost) / (Nr. of Operatons) = 3 Asymptotcally, cost s O() or the same as for a table of fxed sze. Advanced Algorthms, Feodor F. Dragan, Kent State Unversty 9
10 Accountng Analyss Avods math f you can guess charges that work. Charge each operaton 3 unts for amortzed cost: use to perform mmedate nserton store When table doubles, use unt to re-nsert tems added snce last copy. use unt to re-nsert tems coped prevously. Potental Analyss Defne Φ(T) = num(t) - sze(t) Immedately after an expanson but before a new tem s nserted num(t) = sze(t) / whch mples that Φ(T) = 0 Also, when the table s empty, Φ (T) = 0. Snce the table s always half-full Φ(T) 0 Some useful defntons: num = number of elements after th operaton sze = table sze after th operaton φ = potental after th operaton Snce num 0 = sze 0, t follows that φ 0 = 0 If th nserton does not trgger an expanson, then sze = sze - and ĉ = c + φ + φ - = + ( num - sze ) - ( num - - sze - ) = + ( num - sze ) - (num - ) + sze = 3 Advanced Algorthms, Feodor F. Dragan, Kent State Unversty 0
11 Potental Analyss (cont.) If the th nserton causes an expanson, then and ĉ = c + φ - φ - = num + ( num - sze ) - ( num - - sze - ) = num + num - (num - ) - [ (num - ) - (num - )] = 3... sze sze num = = num( T ) Defnton: The load factor α (T ) of a nonempty table T s α ( T ) =. sze( T ) Observatons:. 0 α( T ) and sze( T ) α( T ) = num( T ),. If T s empty, then num(t ) = 0. It s natural and useful to defne for empty table sze(t ) = 0 α(t ) = Note that these defntons preserve the equalty n the frst precedng observaton. Advanced Algorthms, Feodor F. Dragan, Kent State Unversty
12 Dynamc Tables wth Insert and Delete Goal: - Keep the load factor of the dynamc table bound below by a postve constant. - Keep the amortzed cost of each table operaton bounded above by a constant. Proposed Plan: When the table usage drops below ½, we could reduce the table to one-half ts sze. Problem: Ths may cause thrashng. If a table s ntally full, consder the followng operatons: I DD I I DD I I A sequence of n nsertons to fll the table, followed by a sequence of the above n operatons has an average cost of O(n) per operaton. We avod ths problem by watng untl the table s well below ½ full before contractng t. By contractng the table when t falls below ¼ full, we mantan the lower bound α( T ). 4 If the table becomes empty, we cut ts sze to 0. Advanced Algorthms, Feodor F. Dragan, Kent State Unversty
13 Analyss by the Potental Method Goal: We want the potental Φ (T ) to satsfy Φ (T ) = 0 mmedately followng a contracton/expanson (before any element s added/deleted). Φ (T ) bulds to pay for a change as the load factor approaches ¼ or. Defnton: Propertes: num( T ) sze( T ) Φ( T ) = sze( T ) num( T ) f α( T ). f α( T ) < Both branches of formula agree (and equal 0) at ther swtchover pont, α( T ) = If α( T ) =, then num( T ) = sze( T ), Φ( T ) = num( T ). So the potental can pay to move each tem. If α( T ) =, then 4num( T ) = sze( T ), 4 and Φ( T ) = 4num( T ) num( T ) = num( T ). So the potental can pay to move each tem. Φ (T) s zero mmedately after each table expanson or contracton. Exercse: Consder graph of Φ(T) over [8,3] mmedately after num ncreases from 6 to 7. Advanced Algorthms, Feodor F. Dragan, Kent State Unversty 3.
14 Analyss by the Potental Method (INSERT) Notaton: The subscrpt n cˆ, c, num, sze, α, Φ wll denote ther values after the th operaton. Observaton: Recall that ntally num = sze = Φ = ; α = Case : Suppose the th operaton s INSERT and α ( T ). Then the analyss s same as for a table allowng only INSERT and ĉ = 3. Case : Suppose the th operaton s INSERT and α ( T ) <, α ( ). T < ĉ = c + φ - φ - = + (½ sze - num ) - (½ sze - - num - ) = + ½ sze - num - ½ sze + num - = 0. Case 3: Suppose the th operaton s INSERT and α, ( ) ( T ) < α T. = c + φ - φ - = + ( num - sze ) - (½ sze - - num - ) ĉ = + ( (num - +) - sze - ) (½ sze - - num - ) Advanced Algorthms, Feodor F. Dragan, Kent State Unversty 4 = 3 num - ½ (3 sze - )+3= 3 sze - ½ (3 sze - )+3 < 3/ sze - - 3/ sze - )+3 = 3. α Thus, for an INSERT, the amortzed cost s cˆ 3..
15 Analyss by the Potental Method (DELETE) Case 4: Suppose the th operaton s DELETE and α ( T ) <, but a deleton does not cause a contracton. An easy calculaton (n text) shows that cˆ =. Case 5: Suppose α ( T but deleton causes a contracton. ) <, one tem s deleted, leavng num tems to move, so c = + num. note that sze = ½ sze - = num - = (num + ). Then, ĉ = c + φ - φ - = + num + (½ sze - num ) - (½ sze - - num - ) = + num + num + - num num - + num + =. Case 6: α ( T ) and th operaton s a deleton. By Exercse 7.4-, the amortzed cost ĉ wth respect to the potental functon s bounded above by a constant. Exercse 7.4- s assgned as homework. In all cases, the amortzed cost for each operaton s bounded above by a constant. Thus, the actual tme for n nsert and delete operatons on a dynamc table s O(n). READ Ch. 7 n CLRS. Advanced Algorthms, Feodor F. Dragan, Kent State Unversty 5
find (x): given element x, return the canonical element of the set containing x;
COS 43 Sprng, 009 Dsjont Set Unon Problem: Mantan a collecton of dsjont sets. Two operatons: fnd the set contanng a gven element; unte two sets nto one (destructvely). Approach: Canoncal element method:
More informationData Structures and Algorithm. Xiaoqing Zheng
Data Structures and Algorithm Xiaoqing Zheng zhengxq@fudan.edu.cn MULTIPOP top[s] = 6 top[s] = 2 3 2 8 5 6 5 S MULTIPOP(S, x). while not STACK-EMPTY(S) and k 0 2. do POP(S) 3. k k MULTIPOP(S, 4) Analysis
More informationProblem Set 9 Solutions
Desgn and Analyss of Algorthms May 4, 2015 Massachusetts Insttute of Technology 6.046J/18.410J Profs. Erk Demane, Srn Devadas, and Nancy Lynch Problem Set 9 Solutons Problem Set 9 Solutons Ths problem
More informationStructure and Drive Paul A. Jensen Copyright July 20, 2003
Structure and Drve Paul A. Jensen Copyrght July 20, 2003 A system s made up of several operatons wth flow passng between them. The structure of the system descrbes the flow paths from nputs to outputs.
More informationEdge Isoperimetric Inequalities
November 7, 2005 Ross M. Rchardson Edge Isopermetrc Inequaltes 1 Four Questons Recall that n the last lecture we looked at the problem of sopermetrc nequaltes n the hypercube, Q n. Our noton of boundary
More informationGraduate Analysis of Algorithms Dr. Haim Levkowitz
UMass Lowell Computer Science 9.53 Graduate Analysis of Algorithms Dr. Haim Levkowitz Fall 27 Lecture 5 Tuesday, 2 Oct 27 Amortized Analysis Overview Amortize: To pay off a debt, usually by periodic payments
More informationToday: Amortized Analysis
Today: Amortized Analysis COSC 581, Algorithms March 6, 2014 Many of these slides are adapted from several online sources Today s class: Chapter 17 Reading Assignments Reading assignment for next class:
More informationNotes on Frequency Estimation in Data Streams
Notes on Frequency Estmaton n Data Streams In (one of) the data streamng model(s), the data s a sequence of arrvals a 1, a 2,..., a m of the form a j = (, v) where s the dentty of the tem and belongs to
More informationHashing. Alexandra Stefan
Hashng Alexandra Stefan 1 Hash tables Tables Drect access table (or key-ndex table): key => ndex Hash table: key => hash value => ndex Man components Hash functon Collson resoluton Dfferent keys mapped
More informationCalculation of time complexity (3%)
Problem 1. (30%) Calculaton of tme complexty (3%) Gven n ctes, usng exhaust search to see every result takes O(n!). Calculaton of tme needed to solve the problem (2%) 40 ctes:40! dfferent tours 40 add
More informationCS 5321: Advanced Algorithms Amortized Analysis of Data Structures. Motivations. Motivation cont d
CS 5321: Advanced Algorithms Amortized Analysis of Data Structures Ali Ebnenasir Department of Computer Science Michigan Technological University Motivations Why amortized analysis and when? Suppose you
More informationAnother way of saying this is that amortized analysis guarantees the average case performance of each operation in the worst case.
Amortized Analysis: CLRS Chapter 17 Last revised: August 30, 2006 1 In amortized analysis we try to analyze the time required by a sequence of operations. There are many situations in which, even though
More informationLinear Regression Analysis: Terminology and Notation
ECON 35* -- Secton : Basc Concepts of Regresson Analyss (Page ) Lnear Regresson Analyss: Termnology and Notaton Consder the generc verson of the smple (two-varable) lnear regresson model. It s represented
More informationAmortized Analysis (chap. 17)
Amortized Analysis (chap. 17) Not just consider one operation, but a sequence of operations on a given data structure. Average cost over a sequence of operations. Probabilistic analysis: Average case running
More informationIntroduction to Algorithms
Introducton to Algorthms 6.046J/8.40J Lecture 7 Prof. Potr Indyk Data Structures Role of data structures: Encapsulate data Support certan operatons (e.g., INSERT, DELETE, SEARCH) Our focus: effcency of
More informationn α j x j = 0 j=1 has a nontrivial solution. Here A is the n k matrix whose jth column is the vector for all t j=0
MODULE 2 Topcs: Lnear ndependence, bass and dmenson We have seen that f n a set of vectors one vector s a lnear combnaton of the remanng vectors n the set then the span of the set s unchanged f that vector
More informationMath Review. CptS 223 Advanced Data Structures. Larry Holder School of Electrical Engineering and Computer Science Washington State University
Math Revew CptS 223 dvanced Data Structures Larry Holder School of Electrcal Engneerng and Computer Scence Washngton State Unversty 1 Why do we need math n a data structures course? nalyzng data structures
More informationAmortized Analysis. DistributeMoney(n, k) 1 Each of n people gets $1. 2 for i = 1to k 3 do Give a dollar to a random person
Amortized Analysis DistributeMoney(n, k) 1 Each of n people gets $1. 2 for i = 1to k 3 do Give a dollar to a random person What is the maximum amount of money I can receive? Amortized Analysis DistributeMoney(n,
More informationAssortment Optimization under MNL
Assortment Optmzaton under MNL Haotan Song Aprl 30, 2017 1 Introducton The assortment optmzaton problem ams to fnd the revenue-maxmzng assortment of products to offer when the prces of products are fxed.
More informationExercises of Chapter 2
Exercses of Chapter Chuang-Cheh Ln Department of Computer Scence and Informaton Engneerng, Natonal Chung Cheng Unversty, Mng-Hsung, Chay 61, Tawan. Exercse.6. Suppose that we ndependently roll two standard
More information= z 20 z n. (k 20) + 4 z k = 4
Problem Set #7 solutons 7.2.. (a Fnd the coeffcent of z k n (z + z 5 + z 6 + z 7 + 5, k 20. We use the known seres expanson ( n+l ( z l l z n below: (z + z 5 + z 6 + z 7 + 5 (z 5 ( + z + z 2 + z + 5 5
More informationLecture 4: November 17, Part 1 Single Buffer Management
Lecturer: Ad Rosén Algorthms for the anagement of Networs Fall 2003-2004 Lecture 4: November 7, 2003 Scrbe: Guy Grebla Part Sngle Buffer anagement In the prevous lecture we taled about the Combned Input
More informationLecture 7: Amortized analysis
Lecture 7: Amortized analysis In many applications we want to minimize the time for a sequence of operations. To sum worst-case times for single operations can be overly pessimistic, since it ignores correlation
More informationMore metrics on cartesian products
More metrcs on cartesan products If (X, d ) are metrc spaces for 1 n, then n Secton II4 of the lecture notes we defned three metrcs on X whose underlyng topologes are the product topology The purpose of
More informationx = , so that calculated
Stat 4, secton Sngle Factor ANOVA notes by Tm Plachowsk n chapter 8 we conducted hypothess tests n whch we compared a sngle sample s mean or proporton to some hypotheszed value Chapter 9 expanded ths to
More informationU.C. Berkeley CS294: Beyond Worst-Case Analysis Luca Trevisan September 5, 2017
U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 4s Luca Trevsan September 5, 07 Summary of Lecture 4 In whch we ntroduce semdefnte programmng and apply t to Max Cut. Semdefnte Programmng Recall that
More informationA 2D Bounded Linear Program (H,c) 2D Linear Programming
A 2D Bounded Lnear Program (H,c) h 3 v h 8 h 5 c h 4 h h 6 h 7 h 2 2D Lnear Programmng C s a polygonal regon, the ntersecton of n halfplanes. (H, c) s nfeasble, as C s empty. Feasble regon C s unbounded
More informationThe Multiple Classical Linear Regression Model (CLRM): Specification and Assumptions. 1. Introduction
ECONOMICS 5* -- NOTE (Summary) ECON 5* -- NOTE The Multple Classcal Lnear Regresson Model (CLRM): Specfcaton and Assumptons. Introducton CLRM stands for the Classcal Lnear Regresson Model. The CLRM s also
More informationSplit alignment. Martin C. Frith April 13, 2012
Splt algnment Martn C. Frth Aprl 13, 2012 1 Introducton Ths document s about algnng a query sequence to a genome, allowng dfferent parts of the query to match dfferent parts of the genome. Here are some
More informationMin Cut, Fast Cut, Polynomial Identities
Randomzed Algorthms, Summer 016 Mn Cut, Fast Cut, Polynomal Identtes Instructor: Thomas Kesselhem and Kurt Mehlhorn 1 Mn Cuts n Graphs Lecture (5 pages) Throughout ths secton, G = (V, E) s a mult-graph.
More informationLecture 3 January 31, 2017
CS 224: Advanced Algorthms Sprng 207 Prof. Jelan Nelson Lecture 3 January 3, 207 Scrbe: Saketh Rama Overvew In the last lecture we covered Y-fast tres and Fuson Trees. In ths lecture we start our dscusson
More informationEquilibrium with Complete Markets. Instructor: Dmytro Hryshko
Equlbrum wth Complete Markets Instructor: Dmytro Hryshko 1 / 33 Readngs Ljungqvst and Sargent. Recursve Macroeconomc Theory. MIT Press. Chapter 8. 2 / 33 Equlbrum n pure exchange, nfnte horzon economes,
More informationLecture 10 Support Vector Machines II
Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the fake-test data; fxed
More informationEconomics 101. Lecture 4 - Equilibrium and Efficiency
Economcs 0 Lecture 4 - Equlbrum and Effcency Intro As dscussed n the prevous lecture, we wll now move from an envronment where we looed at consumers mang decsons n solaton to analyzng economes full of
More information8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS
SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 493 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces you have studed thus far n the text are real vector spaces because the scalars
More informationSection 8.3 Polar Form of Complex Numbers
80 Chapter 8 Secton 8 Polar Form of Complex Numbers From prevous classes, you may have encountered magnary numbers the square roots of negatve numbers and, more generally, complex numbers whch are the
More informationExpected Value and Variance
MATH 38 Expected Value and Varance Dr. Neal, WKU We now shall dscuss how to fnd the average and standard devaton of a random varable X. Expected Value Defnton. The expected value (or average value, or
More informationFoundations of Arithmetic
Foundatons of Arthmetc Notaton We shall denote the sum and product of numbers n the usual notaton as a 2 + a 2 + a 3 + + a = a, a 1 a 2 a 3 a = a The notaton a b means a dvdes b,.e. ac = b where c s an
More informationMLE and Bayesian Estimation. Jie Tang Department of Computer Science & Technology Tsinghua University 2012
MLE and Bayesan Estmaton Je Tang Department of Computer Scence & Technology Tsnghua Unversty 01 1 Lnear Regresson? As the frst step, we need to decde how we re gong to represent the functon f. One example:
More informationAppendix B: Resampling Algorithms
407 Appendx B: Resamplng Algorthms A common problem of all partcle flters s the degeneracy of weghts, whch conssts of the unbounded ncrease of the varance of the mportance weghts ω [ ] of the partcles
More informationCollege of Computer & Information Science Fall 2009 Northeastern University 20 October 2009
College of Computer & Informaton Scence Fall 2009 Northeastern Unversty 20 October 2009 CS7880: Algorthmc Power Tools Scrbe: Jan Wen and Laura Poplawsk Lecture Outlne: Prmal-dual schema Network Desgn:
More information04 - Treaps. Dr. Alexander Souza
Algorths Theory 04 - Treaps Dr. Alexander Souza The dctonary proble Gven: Unverse (U,
More informationFinding Dense Subgraphs in G(n, 1/2)
Fndng Dense Subgraphs n Gn, 1/ Atsh Das Sarma 1, Amt Deshpande, and Rav Kannan 1 Georga Insttute of Technology,atsh@cc.gatech.edu Mcrosoft Research-Bangalore,amtdesh,annan@mcrosoft.com Abstract. Fndng
More informationLecture 14: Bandits with Budget Constraints
IEOR 8100-001: Learnng and Optmzaton for Sequental Decson Makng 03/07/16 Lecture 14: andts wth udget Constrants Instructor: Shpra Agrawal Scrbed by: Zhpeng Lu 1 Problem defnton In the regular Mult-armed
More informationOnline Classification: Perceptron and Winnow
E0 370 Statstcal Learnng Theory Lecture 18 Nov 8, 011 Onlne Classfcaton: Perceptron and Wnnow Lecturer: Shvan Agarwal Scrbe: Shvan Agarwal 1 Introducton In ths lecture we wll start to study the onlne learnng
More informationDONALD M. DAVIS. 1. Main result
v 1 -PERIODIC 2-EXPONENTS OF SU(2 e ) AND SU(2 e + 1) DONALD M. DAVIS Abstract. We determne precsely the largest v 1 -perodc homotopy groups of SU(2 e ) and SU(2 e +1). Ths gves new results about the largest
More informationOpen Systems: Chemical Potential and Partial Molar Quantities Chemical Potential
Open Systems: Chemcal Potental and Partal Molar Quanttes Chemcal Potental For closed systems, we have derved the followng relatonshps: du = TdS pdv dh = TdS + Vdp da = SdT pdv dg = VdP SdT For open systems,
More informationAffine transformations and convexity
Affne transformatons and convexty The purpose of ths document s to prove some basc propertes of affne transformatons nvolvng convex sets. Here are a few onlne references for background nformaton: http://math.ucr.edu/
More informationLecture 10: May 6, 2013
TTIC/CMSC 31150 Mathematcal Toolkt Sprng 013 Madhur Tulsan Lecture 10: May 6, 013 Scrbe: Wenje Luo In today s lecture, we manly talked about random walk on graphs and ntroduce the concept of graph expander,
More informationLECTURE V. 1. More on the Chinese Remainder Theorem We begin by recalling this theorem, proven in the preceeding lecture.
LECTURE V EDWIN SPARK 1. More on the Chnese Remander Theorem We begn by recallng ths theorem, proven n the preceedng lecture. Theorem 1.1 (Chnese Remander Theorem). Let R be a rng wth deals I 1, I 2,...,
More informationComplete subgraphs in multipartite graphs
Complete subgraphs n multpartte graphs FLORIAN PFENDER Unverstät Rostock, Insttut für Mathematk D-18057 Rostock, Germany Floran.Pfender@un-rostock.de Abstract Turán s Theorem states that every graph G
More informationIntroduction to Vapor/Liquid Equilibrium, part 2. Raoult s Law:
CE304, Sprng 2004 Lecture 4 Introducton to Vapor/Lqud Equlbrum, part 2 Raoult s Law: The smplest model that allows us do VLE calculatons s obtaned when we assume that the vapor phase s an deal gas, and
More informationIntroduction to Algorithms
Introducton to Algorthms 6.046J/18.401J Lecture 7 Prof. Potr Indyk Data Structures Role of data structures: Encapsulate data Support certan operatons (e.g., INSERT, DELETE, SEARCH) What data structures
More informationCS 350 Algorithms and Complexity
CS 350 Algorthms and Complexty Wnter 2015 Lecture 8: Decrease & Conquer (contnued) Andrew P. Black Department of Computer Scence Portland State Unversty Example: DFS traversal of undrected graph a b c
More informationBernoulli Numbers and Polynomials
Bernoull Numbers and Polynomals T. Muthukumar tmk@tk.ac.n 17 Jun 2014 The sum of frst n natural numbers 1, 2, 3,..., n s n n(n + 1 S 1 (n := m = = n2 2 2 + n 2. Ths formula can be derved by notng that
More informationHomework Assignment 3 Due in class, Thursday October 15
Homework Assgnment 3 Due n class, Thursday October 15 SDS 383C Statstcal Modelng I 1 Rdge regresson and Lasso 1. Get the Prostrate cancer data from http://statweb.stanford.edu/~tbs/elemstatlearn/ datasets/prostate.data.
More informationLectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix
Lectures - Week 4 Matrx norms, Condtonng, Vector Spaces, Lnear Independence, Spannng sets and Bass, Null space and Range of a Matrx Matrx Norms Now we turn to assocatng a number to each matrx. We could
More informationE Tail Inequalities. E.1 Markov s Inequality. Non-Lecture E: Tail Inequalities
Algorthms Non-Lecture E: Tal Inequaltes If you hold a cat by the tal you learn thngs you cannot learn any other way. Mar Twan E Tal Inequaltes The smple recursve structure of sp lsts made t relatvely easy
More informationRandomness and Computation
Randomness and Computaton or, Randomzed Algorthms Mary Cryan School of Informatcs Unversty of Ednburgh RC 208/9) Lecture 0 slde Balls n Bns m balls, n bns, and balls thrown unformly at random nto bns usually
More information8.6 The Complex Number System
8.6 The Complex Number System Earler n the chapter, we mentoned that we cannot have a negatve under a square root, snce the square of any postve or negatve number s always postve. In ths secton we want
More informationDesign and Analysis of Algorithms
Desgn and Analyss of Algorthms CSE 53 Lecture 4 Dynamc Programmng Junzhou Huang, Ph.D. Department of Computer Scence and Engneerng CSE53 Desgn and Analyss of Algorthms The General Dynamc Programmng Technque
More informationModule 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Verson ECE IIT, Kharagpur Lesson 6 Theory of Quantzaton Verson ECE IIT, Kharagpur Instructonal Objectves At the end of ths lesson, the students should be able to:
More informationa b a In case b 0, a being divisible by b is the same as to say that
Secton 6.2 Dvsblty among the ntegers An nteger a ε s dvsble by b ε f there s an nteger c ε such that a = bc. Note that s dvsble by any nteger b, snce = b. On the other hand, a s dvsble by only f a = :
More informationErrors for Linear Systems
Errors for Lnear Systems When we solve a lnear system Ax b we often do not know A and b exactly, but have only approxmatons  and ˆb avalable. Then the best thng we can do s to solve ˆx ˆb exactly whch
More informationFeature Selection: Part 1
CSE 546: Machne Learnng Lecture 5 Feature Selecton: Part 1 Instructor: Sham Kakade 1 Regresson n the hgh dmensonal settng How do we learn when the number of features d s greater than the sample sze n?
More informationECE559VV Project Report
ECE559VV Project Report (Supplementary Notes Loc Xuan Bu I. MAX SUM-RATE SCHEDULING: THE UPLINK CASE We have seen (n the presentaton that, for downlnk (broadcast channels, the strategy maxmzng the sum-rate
More informationChapter 9: Statistical Inference and the Relationship between Two Variables
Chapter 9: Statstcal Inference and the Relatonshp between Two Varables Key Words The Regresson Model The Sample Regresson Equaton The Pearson Correlaton Coeffcent Learnng Outcomes After studyng ths chapter,
More informationLecture Space-Bounded Derandomization
Notes on Complexty Theory Last updated: October, 2008 Jonathan Katz Lecture Space-Bounded Derandomzaton 1 Space-Bounded Derandomzaton We now dscuss derandomzaton of space-bounded algorthms. Here non-trval
More information20. Mon, Oct. 13 What we have done so far corresponds roughly to Chapters 2 & 3 of Lee. Now we turn to Chapter 4. The first idea is connectedness.
20. Mon, Oct. 13 What we have done so far corresponds roughly to Chapters 2 & 3 of Lee. Now we turn to Chapter 4. The frst dea s connectedness. Essentally, we want to say that a space cannot be decomposed
More informationOutline and Reading. Dynamic Programming. Dynamic Programming revealed. Computing Fibonacci. The General Dynamic Programming Technique
Outlne and Readng Dynamc Programmng The General Technque ( 5.3.2) -1 Knapsac Problem ( 5.3.3) Matrx Chan-Product ( 5.3.1) Dynamc Programmng verson 1.4 1 Dynamc Programmng verson 1.4 2 Dynamc Programmng
More informationLecture 21: Numerical methods for pricing American type derivatives
Lecture 21: Numercal methods for prcng Amercan type dervatves Xaoguang Wang STAT 598W Aprl 10th, 2014 (STAT 598W) Lecture 21 1 / 26 Outlne 1 Fnte Dfference Method Explct Method Penalty Method (STAT 598W)
More informationLecture 4: Universal Hash Functions/Streaming Cont d
CSE 5: Desgn and Analyss of Algorthms I Sprng 06 Lecture 4: Unversal Hash Functons/Streamng Cont d Lecturer: Shayan Oves Gharan Aprl 6th Scrbe: Jacob Schreber Dsclamer: These notes have not been subjected
More informationFREQUENCY DISTRIBUTIONS Page 1 of The idea of a frequency distribution for sets of observations will be introduced,
FREQUENCY DISTRIBUTIONS Page 1 of 6 I. Introducton 1. The dea of a frequency dstrbuton for sets of observatons wll be ntroduced, together wth some of the mechancs for constructng dstrbutons of data. Then
More informationREAL ANALYSIS I HOMEWORK 1
REAL ANALYSIS I HOMEWORK CİHAN BAHRAN The questons are from Tao s text. Exercse 0.0.. If (x α ) α A s a collecton of numbers x α [0, + ] such that x α
More informationThermodynamics Second Law Entropy
Thermodynamcs Second Law Entropy Lana Sherdan De Anza College May 8, 2018 Last tme the Boltzmann dstrbuton (dstrbuton of energes) the Maxwell-Boltzmann dstrbuton (dstrbuton of speeds) the Second Law of
More informationTHE SUMMATION NOTATION Ʃ
Sngle Subscrpt otaton THE SUMMATIO OTATIO Ʃ Most of the calculatons we perform n statstcs are repettve operatons on lsts of numbers. For example, we compute the sum of a set of numbers, or the sum of the
More informationNUMERICAL DIFFERENTIATION
NUMERICAL DIFFERENTIATION 1 Introducton Dfferentaton s a method to compute the rate at whch a dependent output y changes wth respect to the change n the ndependent nput x. Ths rate of change s called the
More informationPortfolios with Trading Constraints and Payout Restrictions
Portfolos wth Tradng Constrants and Payout Restrctons John R. Brge Northwestern Unversty (ont wor wth Chrs Donohue Xaodong Xu and Gongyun Zhao) 1 General Problem (Very) long-term nvestor (eample: unversty
More information3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X
Statstcs 1: Probablty Theory II 37 3 EPECTATION OF SEVERAL RANDOM VARIABLES As n Probablty Theory I, the nterest n most stuatons les not on the actual dstrbuton of a random vector, but rather on a number
More informationCOMPARISON OF SOME RELIABILITY CHARACTERISTICS BETWEEN REDUNDANT SYSTEMS REQUIRING SUPPORTING UNITS FOR THEIR OPERATIONS
Avalable onlne at http://sck.org J. Math. Comput. Sc. 3 (3), No., 6-3 ISSN: 97-537 COMPARISON OF SOME RELIABILITY CHARACTERISTICS BETWEEN REDUNDANT SYSTEMS REQUIRING SUPPORTING UNITS FOR THEIR OPERATIONS
More informationLecture 3: Probability Distributions
Lecture 3: Probablty Dstrbutons Random Varables Let us begn by defnng a sample space as a set of outcomes from an experment. We denote ths by S. A random varable s a functon whch maps outcomes nto the
More information2.3 Nilpotent endomorphisms
s a block dagonal matrx, wth A Mat dm U (C) In fact, we can assume that B = B 1 B k, wth B an ordered bass of U, and that A = [f U ] B, where f U : U U s the restrcton of f to U 40 23 Nlpotent endomorphsms
More informationLecture Notes on Linear Regression
Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume
More informationChapter Newton s Method
Chapter 9. Newton s Method After readng ths chapter, you should be able to:. Understand how Newton s method s dfferent from the Golden Secton Search method. Understand how Newton s method works 3. Solve
More informationTHE CHINESE REMAINDER THEOREM. We should thank the Chinese for their wonderful remainder theorem. Glenn Stevens
THE CHINESE REMAINDER THEOREM KEITH CONRAD We should thank the Chnese for ther wonderful remander theorem. Glenn Stevens 1. Introducton The Chnese remander theorem says we can unquely solve any par of
More information18.1 Introduction and Recap
CS787: Advanced Algorthms Scrbe: Pryananda Shenoy and Shjn Kong Lecturer: Shuch Chawla Topc: Streamng Algorthmscontnued) Date: 0/26/2007 We contnue talng about streamng algorthms n ths lecture, ncludng
More informationFinding Primitive Roots Pseudo-Deterministically
Electronc Colloquum on Computatonal Complexty, Report No 207 (205) Fndng Prmtve Roots Pseudo-Determnstcally Ofer Grossman December 22, 205 Abstract Pseudo-determnstc algorthms are randomzed search algorthms
More informationRandom Walks on Digraphs
Random Walks on Dgraphs J. J. P. Veerman October 23, 27 Introducton Let V = {, n} be a vertex set and S a non-negatve row-stochastc matrx (.e. rows sum to ). V and S defne a dgraph G = G(V, S) and a drected
More informationMath 217 Fall 2013 Homework 2 Solutions
Math 17 Fall 013 Homework Solutons Due Thursday Sept. 6, 013 5pm Ths homework conssts of 6 problems of 5 ponts each. The total s 30. You need to fully justfy your answer prove that your functon ndeed has
More informationModule 9. Lecture 6. Duality in Assignment Problems
Module 9 1 Lecture 6 Dualty n Assgnment Problems In ths lecture we attempt to answer few other mportant questons posed n earler lecture for (AP) and see how some of them can be explaned through the concept
More informationThe Minimum Universal Cost Flow in an Infeasible Flow Network
Journal of Scences, Islamc Republc of Iran 17(2): 175-180 (2006) Unversty of Tehran, ISSN 1016-1104 http://jscencesutacr The Mnmum Unversal Cost Flow n an Infeasble Flow Network H Saleh Fathabad * M Bagheran
More informationHMMT February 2016 February 20, 2016
HMMT February 016 February 0, 016 Combnatorcs 1. For postve ntegers n, let S n be the set of ntegers x such that n dstnct lnes, no three concurrent, can dvde a plane nto x regons (for example, S = {3,
More informationMaximizing Overlap of Large Primary Sampling Units in Repeated Sampling: A comparison of Ernst s Method with Ohlsson s Method
Maxmzng Overlap of Large Prmary Samplng Unts n Repeated Samplng: A comparson of Ernst s Method wth Ohlsson s Method Red Rottach and Padrac Murphy 1 U.S. Census Bureau 4600 Slver Hll Road, Washngton DC
More informationExample: (13320, 22140) =? Solution #1: The divisors of are 1, 2, 3, 4, 5, 6, 9, 10, 12, 15, 18, 20, 27, 30, 36, 41,
The greatest common dvsor of two ntegers a and b (not both zero) s the largest nteger whch s a common factor of both a and b. We denote ths number by gcd(a, b), or smply (a, b) when there s no confuson
More informationCHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE
CHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE Analytcal soluton s usually not possble when exctaton vares arbtrarly wth tme or f the system s nonlnear. Such problems can be solved by numercal tmesteppng
More informationprinceton univ. F 13 cos 521: Advanced Algorithm Design Lecture 3: Large deviations bounds and applications Lecturer: Sanjeev Arora
prnceton unv. F 13 cos 521: Advanced Algorthm Desgn Lecture 3: Large devatons bounds and applcatons Lecturer: Sanjeev Arora Scrbe: Today s topc s devaton bounds: what s the probablty that a random varable
More informationLecture 12: Discrete Laplacian
Lecture 12: Dscrete Laplacan Scrbe: Tanye Lu Our goal s to come up wth a dscrete verson of Laplacan operator for trangulated surfaces, so that we can use t n practce to solve related problems We are mostly
More informationCS473-Algorithms I. Lecture 12b. Dynamic Tables. CS 473 Lecture X 1
CS473-Algorthm I Lecture b Dyamc Table CS 473 Lecture X Why Dyamc Table? I ome applcato: We do't kow how may object wll be tored a table. We may allocate pace for a table But, later we may fd out that
More informationGeneral theory of fuzzy connectedness segmentations: reconciliation of two tracks of FC theory
General theory of fuzzy connectedness segmentatons: reconclaton of two tracks of FC theory Krzysztof Chrs Ceselsk Department of Mathematcs, West Vrgna Unversty and MIPG, Department of Radology, Unversty
More informationPolynomials. 1 More properties of polynomials
Polynomals 1 More propertes of polynomals Recall that, for R a commutatve rng wth unty (as wth all rngs n ths course unless otherwse noted), we defne R[x] to be the set of expressons n =0 a x, where a
More information