Outline and Reading. Dynamic Programming. Dynamic Programming revealed. Computing Fibonacci. The General Dynamic Programming Technique
|
|
- Buddy Burns
- 5 years ago
- Views:
Transcription
1 Outlne and Readng Dynamc Programmng The General Technque ( 5.3.2) -1 Knapsac Problem ( 5.3.3) Matrx Chan-Product ( 5.3.1) Dynamc Programmng verson Dynamc Programmng verson Dynamc Programmng revealed Brea problem nto subproblems that are shared have subproblem optmalty (optmal subproblem soluton helps solve overall problem) subproblem optmalty means can wrte recursve realtonshp between subproblems! Defnng subproblems s hardest part! Compute solutons to small subproblems Store solutons n array A. Combne already computed solutons nto solutons for larger subproblems Solutons Array A s teratvely flled (Optonal: reduce space needed by reusng array) Dynamc Programmng verson Computng Fbonacc Dynamc Programmng s a general algorthm desgn paradgm: Iteratvely solves small subproblems whch are combned to solve overall problem. Fbonacc numbers defned F = F 1 = 1 F n = F n-1 + F n-2, for n > 1 Recursve soluton: nt fb(nt x) f (x=) return ; f (x=1) return 1; return fb(x-1) + fb(x-2); Dynamc Programmng Soluton: f[]=; f[1]=1; for 2 to x do f[] f[-1] + f[-2]; return f[x]; Dynamc Programmng verson Reducng Space for Computng Fbonacc store only prevous 2 values to compute next value nt fb(x) f (x=) return ; f (x=1) return 1; nt last 1; nextlast ; for 2 to x do temp last + nextlast; nextlast last; last temp; return temp; The General Dynamc Programmng Technque Apples to a problem that at frst seems to requre a lot of tme (possbly exponental), provded we have: Smple subproblems: the subproblems can be defned n terms of a few varables, such as, l, m, and so on. Subproblem optmalty: the global optmum value can be defned n terms of optmal subproblems Subproblem overlap: the subproblems are not ndependent, but nstead they overlap (hence, should be constructed bottom-up). Dynamc Programmng verson Dynamc Programmng verson 1.4 6
2 The /1 Knapsac Problem Gven: A set S of n tems, wth each tem havng b - a postve beneft w - a postve weght Goal: Choose tems wth maxmum total beneft but wth. If we are not allowed to tae fractonal amounts, then ths s the /1 napsac problem. In ths case, we let T denote the set of tems we tae Obectve: maxmze Constrant: w W b T T Dynamc Programmng verson Example Items: Gven: A set S of n tems, wth each tem havng b - a postve beneft w - a postve weght Goal: Choose tems wth maxmum total beneft but wth. napsac Weght: Beneft: n 2 n 2 n 6 n 2 n $2 $3 $6 $25 $8 9 n Soluton: 5 (2 n) 3 (2 n) 1 (4 n) Dynamc Programmng verson A /1 Knapsac, Frst Attempt S : Set of tems numbered 1 to. Defne ] = best selecton from S. Problem: does not have subproblem optmalty: Consder S={(3,2),(5,4),(8,5),(4,3),(1,9)} beneft-weght pars Best for S 4 : Best for S 5 : Dynamc Programmng verson A /1 Knapsac, Second Attempt S : Set of tems numbered 1 to. Defne = best selecton from S wth weght exactly equal to w Good news: ths does have subproblem optmalty: >, w w } I.e., best subset of S wth weght lmt exactly w s ether the best subset of S -1 w/ weght w or the best subset of S -1 w/ weght w-w plus beneft of tem. Dynamc Programmng verson w Towards the /1 Knapsac S : Set of tems numbered 1 to = {(b 1,w 1 ), (b 2,w 2 ),, (b,w )} Defne = maxmum beneft of optmal subset from S wth total weght at most Recursve defnton of :, w } f = otherwse Dynamc Programmng verson Towards the /1 Knapsac f =, w } otherwse rec1knap(s, W): = maxmum beneft Input: set S of tems w/ beneft b 1, b 2, b of optmal subset from S,; weghts w 1, w 2, w and max. weght W wth total weght at most Output: beneft of best subset wth Recursve verson of f = then {S = emptyset} algorthm based on return recursve subproblem remove tem (beneft-weght (b,w )) relatonshp. from S Not a dynamc f w > W then {tem does not ft} return rec1knap(s,w) programmng verson. return max(rec1knap(s,w), rec1knap(s,w-w ) + b ) Dynamc Programmng verson
3 Towards the /1 Knapsac Modfed recursve verson that stores subproblem solutons Frst allocate global array B of sze n+1 by W Then ntalze all entres of, to 1 B stores results of recursve calls Entres n B are computed when necessary Ths s consdered a dynamc programmng verson., w ] + b } f = otherwse rec1knap(s, W): Input: set S of tems w/ beneft b 1, b 2,,b,; weghts w 1, w 2, w and max. weght W Output: beneft of best subset wth f = then return remove tem (beneft-weght (b,w )) from S f -1, W]= 1 then -1,W]=rec1Knap(S,W) f w > W then return -1, W] f -1, W- w ]= 1 then -1,W - w ]=rec1knap(s,w -w ) return max(-1, W], -1,W - w ]+b ) Dynamc Programmng verson The /1 Knapsac - Iteratve f =, w } otherwse 1Knapsac(S, W): Input: set S of n tems w/ beneft b Recursve computaton and weght w ; max. weght W not necessary Output: beneft of best subset wth Compute teratvely, for w to W do {base case} bottom-up, All -1,*] must be for 1 to n do computed before *] for 1 to W do because of subproblem f w then dependences -1, Ths s also dynamc max(-1,, programmng. -1,-w Dynamc Programmng verson 1.4 ]+b ) 14 The /1 Knapsac - Iteratve The /1 Knapsac - Iteratve f = f =, w } otherwse, w } otherwse 1Knapsac(S, W): 1Knapsac(S, W): Input: set S of n tems w/ beneft b Input: set S of n tems w/ beneft b Not necessary to use all the and weght w ; max. weght W Not necessary to use all the and weght w space ; max. weght W space Output: beneft of best subset wth Output: beneft of best subset wth Keep trac of one row at a Keep trac of one row at a tme for w to W do {base case} tme for w to W do {base case} Overwrte results from, Overwrte results from prevous row as new values for 1 to n do prevous row as new values for 1 to n do computed for W downto 1 do computed for W downto 1 do Must compute rght to left (W f w then Must compute rght to left (W f w then downto 1) so that the next -1, downto 1) so that the next row (*]) uses results from row (*]) uses results from the prevous row (-1,*]). the prevous row (-1,*]). max(-1,, max(, Smplfy ths to get verson n -1, -w ]+b ) Smplfy ths to get verson n -w ]+b ) boo. Dynamc Programmng verson boo. Dynamc Programmng verson The /1 Knapsac > w, w w } 1Knapsac(S, W): The boo verson: Input: set S of n tems w/ beneft b When value does not change and weght w ; max. weght W from one row to the next, Output: beneft of best subset wth then no need to assgn same value. for w to W do Runnng tme: O(nW). Not a polynomal-tme for 1 to n do algorthm f W s large for w W downto w do Ths s a pseudo-polynomal f w-w ]+b > then tme algorthm w-w ]+b Dynamc Programmng verson lne-breang problem Gven sequence of words from one paragraph Return where lne-breas should occur Mnmze empty space on each lne (except for last lne of paragraph) Dynamc Programmng verson
4 lne-breang problem Example problem A smple verson: letters and spaces have equal wdth nput s set of n word lengths, w 1, w 2, w n also gven lne wdth lmt L. each length w ncludes one space Placng words up to on one lne means w L = w = Penalty for extra spaces X = L s X 3 Mnmze sum of penaltes from each lne (no last lne penalty) Dynamc Programmng verson Paragraph s: Those who cannot remember the past are condemned to repeat t. Word lengths are 6,4,7,9,4,5,4,1,3,7,4. Suppose lne wdth L = 17. Fnd an optmal way of separatng words nto lnes that mnmzes penalty. Dynamc Programmng verson lnebrea DP lnebrea DP cost for n-1 downto do f (w[] + w[+1] + + w[n-1] < L) lne] ; mncost Infnty; 1; whle ( words startng from w[] ft on a lne) // meanng (w[] + w[+1] + + w[+-1] <= L) lnecost penalty from placng words w[] to w[+-1] on one lne. totalcost lnecost + lne+]; mncost mn(totalcost, mncost) // trac mn. so far ++; lne]=mncost; O(nL); L s maxmum wdth Lnear f L s consdered constant Space O(n). Dynamc Programmng verson Dynamc Programmng verson Matrx Chan-Products Revew: Matrx Multplcaton. C = A*B A s d e and B s e f e 1 = C[, A[, ]* = O(def ) tme (def multplcatons) d A e e B C f, d Matrx Chan-Products Matrx Chan-Product: Compute A=A *A 1 * *A n-1 A s d d +1 Problem: How to parenthesze? [for mnmzng ops] Example B s 3 1 C s 1 5 D s 5 5 (B*C)*D taes = 1575 ops B*(C*D) taes = 4 ops Dynamc Programmng verson f Dynamc Programmng verson
5 An Enumeraton Approach Matrx Chan-Product Alg.: Try all possble ways to parenthesze A=A *A 1 * *A n-1 Calculate number of ops for each one Pc the one that s best Runnng tme: The number of paranetheszatons s equal to the number of bnary trees wth n nodes Ths s exponental! It s called the Catalan number, and t s almost 4 n. Ths s a terrble algorthm! Dynamc Programmng verson A Greedy Approach Idea #1: repeatedly select the product that uses (up) the most operatons. Counter-example: A s 1 5 B s 5 1 C s 1 5 D s 5 1 Greedy dea #1 gves (A*B)*(C*D), whch taes = 2 ops A*((B*C)*D) taes = 1 ops Dynamc Programmng verson Another Greedy Approach Idea #2: repeatedly select the product that uses the fewest operatons. Counter-example: A s B s 11 9 C s 9 1 D s 1 99 Greedy dea #2 gves A*((B*C)*D)), whch taes = ops (A*B)*(C*D) taes =1899 ops The greedy approach s not gvng us the A Recursve Approach Defne subproblems: Fnd the best parentheszaton of A *A +1 * *A. Let N, denote the number of operatons done by ths subproblem. The optmal soluton for the whole problem s N,n-1. Subproblem optmalty: The optmal soluton can be defned n terms of optmal subproblems There has to be a fnal multplcaton (root of the expresson tree) for the optmal soluton. Say, the fnal multply s at ndex : (A * *A )*(A +1 * *A n-1 ). Then the optmal soluton N,n-1 s the sum of two optmal subproblems, N, and N +1,n-1 plus the tme for the last multply. If subproblems were not optmal, nether s global soluton. optmal value. Dynamc Programmng verson Dynamc Programmng verson A Characterzng Equaton Defne global optmal n terms of optmal subproblems, by checng all possble locatons for fnal multply. Recall that A s a d d +1 dmensonal matrx. So, a characterzng equaton for N, s the followng: N, = + < mn{ N, + N + 1, + dd+ 1d 1} Note that subproblems are not ndependent--the subproblems overlap (are shared) A Dynamc Programmng Construct optmal subproblems bottom-up. N, s are easy, so start wth them Then do length 2,3, subproblems, and so on. Array N, stores solutons Runnng tme: O(n 3 ) matrxchan(s): Input: sequence S of n matrces to be multpled Output: number of operatons n an optmal parantheszaton of S for 1 to n-1 do N, for b 1 to n-1 do for to n-b-1 do +b N, +nfnty for to -1 do N, mn{n,, N, +N +1, +d d +1 d +1 } Dynamc Programmng verson Dynamc Programmng verson 1.4 3
6 A Dynamc Programmng Vsualzaton N The bottom-up constructon flls n the N array by dagonals N, gets values from pervous entres n -th row and -th column Fllng n each entry n the N table taes O(n) tme. Total run tme: O(n 3 ) Gettng actual parentheszaton can be done by rememberng for each N entry, = + mn{ N, + N+ 1, + dd+ 1d 1} < answer N n-1 n-1 Dynamc Programmng verson
Dynamic Programming. Preview. Dynamic Programming. Dynamic Programming. Dynamic Programming (Example: Fibonacci Sequence)
/24/27 Prevew Fbonacc Sequence Longest Common Subsequence Dynamc programmng s a method for solvng complex problems by breakng them down nto smpler sub-problems. It s applcable to problems exhbtng the propertes
More informationProblem Set 9 Solutions
Desgn and Analyss of Algorthms May 4, 2015 Massachusetts Insttute of Technology 6.046J/18.410J Profs. Erk Demane, Srn Devadas, and Nancy Lynch Problem Set 9 Solutons Problem Set 9 Solutons Ths problem
More informationDynamic Programming 4/5/12. Dynamic programming. Fibonacci numbers. Fibonacci: a first attempt. David Kauchak cs302 Spring 2012
Dynamc Programmng Davd Kauchak cs32 Sprng 212 Dynamc programmng l One of the most mportant algorthm tools! l Very common ntervew queston l Method for solvng problems where optmal solutons can be defned
More informationDesign and Analysis of Algorithms
Desgn and Analyss of Algorthms CSE 53 Lecture 4 Dynamc Programmng Junzhou Huang, Ph.D. Department of Computer Scence and Engneerng CSE53 Desgn and Analyss of Algorthms The General Dynamc Programmng Technque
More informationDynamic Programming! CSE 417: Algorithms and Computational Complexity!
Dynamc Programmng CSE 417: Algorthms and Computatonal Complexty Wnter 2009 W. L. Ruzzo Dynamc Programmng, I:" Fbonacc & Stamps Outlne: General Prncples Easy Examples Fbonacc, Lckng Stamps Meater examples
More informationCS 331 DESIGN AND ANALYSIS OF ALGORITHMS DYNAMIC PROGRAMMING. Dr. Daisy Tang
CS DESIGN ND NLYSIS OF LGORITHMS DYNMIC PROGRMMING Dr. Dasy Tang Dynamc Programmng Idea: Problems can be dvded nto stages Soluton s a sequence o decsons and the decson at the current stage s based on the
More informationLectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix
Lectures - Week 4 Matrx norms, Condtonng, Vector Spaces, Lnear Independence, Spannng sets and Bass, Null space and Range of a Matrx Matrx Norms Now we turn to assocatng a number to each matrx. We could
More informationLecture 21: Numerical methods for pricing American type derivatives
Lecture 21: Numercal methods for prcng Amercan type dervatves Xaoguang Wang STAT 598W Aprl 10th, 2014 (STAT 598W) Lecture 21 1 / 26 Outlne 1 Fnte Dfference Method Explct Method Penalty Method (STAT 598W)
More informationCollege of Computer & Information Science Fall 2009 Northeastern University 20 October 2009
College of Computer & Informaton Scence Fall 2009 Northeastern Unversty 20 October 2009 CS7880: Algorthmc Power Tools Scrbe: Jan Wen and Laura Poplawsk Lecture Outlne: Prmal-dual schema Network Desgn:
More informationFeature Selection: Part 1
CSE 546: Machne Learnng Lecture 5 Feature Selecton: Part 1 Instructor: Sham Kakade 1 Regresson n the hgh dmensonal settng How do we learn when the number of features d s greater than the sample sze n?
More informationSection 8.3 Polar Form of Complex Numbers
80 Chapter 8 Secton 8 Polar Form of Complex Numbers From prevous classes, you may have encountered magnary numbers the square roots of negatve numbers and, more generally, complex numbers whch are the
More informationU.C. Berkeley CS294: Beyond Worst-Case Analysis Luca Trevisan September 5, 2017
U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 4s Luca Trevsan September 5, 07 Summary of Lecture 4 In whch we ntroduce semdefnte programmng and apply t to Max Cut. Semdefnte Programmng Recall that
More informationKernel Methods and SVMs Extension
Kernel Methods and SVMs Extenson The purpose of ths document s to revew materal covered n Machne Learnng 1 Supervsed Learnng regardng support vector machnes (SVMs). Ths document also provdes a general
More information8.6 The Complex Number System
8.6 The Complex Number System Earler n the chapter, we mentoned that we cannot have a negatve under a square root, snce the square of any postve or negatve number s always postve. In ths secton we want
More informationFor now, let us focus on a specific model of neurons. These are simplified from reality but can achieve remarkable results.
Neural Networks : Dervaton compled by Alvn Wan from Professor Jtendra Malk s lecture Ths type of computaton s called deep learnng and s the most popular method for many problems, such as computer vson
More informationp 1 c 2 + p 2 c 2 + p 3 c p m c 2
Where to put a faclty? Gven locatons p 1,..., p m n R n of m houses, want to choose a locaton c n R n for the fre staton. Want c to be as close as possble to all the house. We know how to measure dstance
More informationSingular Value Decomposition: Theory and Applications
Sngular Value Decomposton: Theory and Applcatons Danel Khashab Sprng 2015 Last Update: March 2, 2015 1 Introducton A = UDV where columns of U and V are orthonormal and matrx D s dagonal wth postve real
More informationExercises. 18 Algorithms
18 Algorthms Exercses 0.1. In each of the followng stuatons, ndcate whether f = O(g), or f = Ω(g), or both (n whch case f = Θ(g)). f(n) g(n) (a) n 100 n 200 (b) n 1/2 n 2/3 (c) 100n + log n n + (log n)
More informationStructure and Drive Paul A. Jensen Copyright July 20, 2003
Structure and Drve Paul A. Jensen Copyrght July 20, 2003 A system s made up of several operatons wth flow passng between them. The structure of the system descrbes the flow paths from nputs to outputs.
More informationLecture 2: Prelude to the big shrink
Lecture 2: Prelude to the bg shrnk Last tme A slght detour wth vsualzaton tools (hey, t was the frst day... why not start out wth somethng pretty to look at?) Then, we consdered a smple 120a-style regresson
More informationGeneralized Linear Methods
Generalzed Lnear Methods 1 Introducton In the Ensemble Methods the general dea s that usng a combnaton of several weak learner one could make a better learner. More formally, assume that we have a set
More informationCalculation of time complexity (3%)
Problem 1. (30%) Calculaton of tme complexty (3%) Gven n ctes, usng exhaust search to see every result takes O(n!). Calculaton of tme needed to solve the problem (2%) 40 ctes:40! dfferent tours 40 add
More informationMath Review. CptS 223 Advanced Data Structures. Larry Holder School of Electrical Engineering and Computer Science Washington State University
Math Revew CptS 223 dvanced Data Structures Larry Holder School of Electrcal Engneerng and Computer Scence Washngton State Unversty 1 Why do we need math n a data structures course? nalyzng data structures
More informationLecture 10 Support Vector Machines II
Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the fake-test data; fxed
More informationSome Comments on Accelerating Convergence of Iterative Sequences Using Direct Inversion of the Iterative Subspace (DIIS)
Some Comments on Acceleratng Convergence of Iteratve Sequences Usng Drect Inverson of the Iteratve Subspace (DIIS) C. Davd Sherrll School of Chemstry and Bochemstry Georga Insttute of Technology May 1998
More informationEEL 6266 Power System Operation and Control. Chapter 3 Economic Dispatch Using Dynamic Programming
EEL 6266 Power System Operaton and Control Chapter 3 Economc Dspatch Usng Dynamc Programmng Pecewse Lnear Cost Functons Common practce many utltes prefer to represent ther generator cost functons as sngle-
More informationExample: (13320, 22140) =? Solution #1: The divisors of are 1, 2, 3, 4, 5, 6, 9, 10, 12, 15, 18, 20, 27, 30, 36, 41,
The greatest common dvsor of two ntegers a and b (not both zero) s the largest nteger whch s a common factor of both a and b. We denote ths number by gcd(a, b), or smply (a, b) when there s no confuson
More informationLecture Notes on Linear Regression
Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume
More informationWeek 5: Neural Networks
Week 5: Neural Networks Instructor: Sergey Levne Neural Networks Summary In the prevous lecture, we saw how we can construct neural networks by extendng logstc regresson. Neural networks consst of multple
More informationNP-Completeness : Proofs
NP-Completeness : Proofs Proof Methods A method to show a decson problem Π NP-complete s as follows. (1) Show Π NP. (2) Choose an NP-complete problem Π. (3) Show Π Π. A method to show an optmzaton problem
More informationModule 9. Lecture 6. Duality in Assignment Problems
Module 9 1 Lecture 6 Dualty n Assgnment Problems In ths lecture we attempt to answer few other mportant questons posed n earler lecture for (AP) and see how some of them can be explaned through the concept
More informationComputing Correlated Equilibria in Multi-Player Games
Computng Correlated Equlbra n Mult-Player Games Chrstos H. Papadmtrou Presented by Zhanxang Huang December 7th, 2005 1 The Author Dr. Chrstos H. Papadmtrou CS professor at UC Berkley (taught at Harvard,
More informationFinding Dense Subgraphs in G(n, 1/2)
Fndng Dense Subgraphs n Gn, 1/ Atsh Das Sarma 1, Amt Deshpande, and Rav Kannan 1 Georga Insttute of Technology,atsh@cc.gatech.edu Mcrosoft Research-Bangalore,amtdesh,annan@mcrosoft.com Abstract. Fndng
More informationprinceton univ. F 17 cos 521: Advanced Algorithm Design Lecture 7: LP Duality Lecturer: Matt Weinberg
prnceton unv. F 17 cos 521: Advanced Algorthm Desgn Lecture 7: LP Dualty Lecturer: Matt Wenberg Scrbe: LP Dualty s an extremely useful tool for analyzng structural propertes of lnear programs. Whle there
More informationSolutions to exam in SF1811 Optimization, Jan 14, 2015
Solutons to exam n SF8 Optmzaton, Jan 4, 25 3 3 O------O -4 \ / \ / The network: \/ where all lnks go from left to rght. /\ / \ / \ 6 O------O -5 2 4.(a) Let x = ( x 3, x 4, x 23, x 24 ) T, where the varable
More informationVector Norms. Chapter 7 Iterative Techniques in Matrix Algebra. Cauchy-Bunyakovsky-Schwarz Inequality for Sums. Distances. Convergence.
Vector Norms Chapter 7 Iteratve Technques n Matrx Algebra Per-Olof Persson persson@berkeley.edu Department of Mathematcs Unversty of Calforna, Berkeley Math 128B Numercal Analyss Defnton A vector norm
More information10-701/ Machine Learning, Fall 2005 Homework 3
10-701/15-781 Machne Learnng, Fall 2005 Homework 3 Out: 10/20/05 Due: begnnng of the class 11/01/05 Instructons Contact questons-10701@autonlaborg for queston Problem 1 Regresson and Cross-valdaton [40
More informationAPPROXIMATE PRICES OF BASKET AND ASIAN OPTIONS DUPONT OLIVIER. Premia 14
APPROXIMAE PRICES OF BASKE AND ASIAN OPIONS DUPON OLIVIER Prema 14 Contents Introducton 1 1. Framewor 1 1.1. Baset optons 1.. Asan optons. Computng the prce 3. Lower bound 3.1. Closed formula for the prce
More informationEEE 241: Linear Systems
EEE : Lnear Systems Summary #: Backpropagaton BACKPROPAGATION The perceptron rule as well as the Wdrow Hoff learnng were desgned to tran sngle layer networks. They suffer from the same dsadvantage: they
More information2.3 Nilpotent endomorphisms
s a block dagonal matrx, wth A Mat dm U (C) In fact, we can assume that B = B 1 B k, wth B an ordered bass of U, and that A = [f U ] B, where f U : U U s the restrcton of f to U 40 23 Nlpotent endomorphsms
More informationCOS 521: Advanced Algorithms Game Theory and Linear Programming
COS 521: Advanced Algorthms Game Theory and Lnear Programmng Moses Charkar February 27, 2013 In these notes, we ntroduce some basc concepts n game theory and lnear programmng (LP). We show a connecton
More informationSection 3.6 Complex Zeros
04 Chapter Secton 6 Comple Zeros When fndng the zeros of polynomals, at some pont you're faced wth the problem Whle there are clearly no real numbers that are solutons to ths equaton, leavng thngs there
More informationSome Consequences. Example of Extended Euclidean Algorithm. The Fundamental Theorem of Arithmetic, II. Characterizing the GCD and LCM
Example of Extended Eucldean Algorthm Recall that gcd(84, 33) = gcd(33, 18) = gcd(18, 15) = gcd(15, 3) = gcd(3, 0) = 3 We work backwards to wrte 3 as a lnear combnaton of 84 and 33: 3 = 18 15 [Now 3 s
More informationFTCS Solution to the Heat Equation
FTCS Soluton to the Heat Equaton ME 448/548 Notes Gerald Recktenwald Portland State Unversty Department of Mechancal Engneerng gerry@pdx.edu ME 448/548: FTCS Soluton to the Heat Equaton Overvew 1. Use
More information1 GSW Iterative Techniques for y = Ax
1 for y = A I m gong to cheat here. here are a lot of teratve technques that can be used to solve the general case of a set of smultaneous equatons (wrtten n the matr form as y = A), but ths chapter sn
More informationAPPENDIX A Some Linear Algebra
APPENDIX A Some Lnear Algebra The collecton of m, n matrces A.1 Matrces a 1,1,..., a 1,n A = a m,1,..., a m,n wth real elements a,j s denoted by R m,n. If n = 1 then A s called a column vector. Smlarly,
More informationMultilayer Perceptron (MLP)
Multlayer Perceptron (MLP) Seungjn Cho Department of Computer Scence and Engneerng Pohang Unversty of Scence and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjn@postech.ac.kr 1 / 20 Outlne
More informationMMA and GCMMA two methods for nonlinear optimization
MMA and GCMMA two methods for nonlnear optmzaton Krster Svanberg Optmzaton and Systems Theory, KTH, Stockholm, Sweden. krlle@math.kth.se Ths note descrbes the algorthms used n the author s 2007 mplementatons
More informationLecture 11. minimize. c j x j. j=1. 1 x j 0 j. +, b R m + and c R n +
Topcs n Theoretcal Computer Scence May 4, 2015 Lecturer: Ola Svensson Lecture 11 Scrbes: Vncent Eggerlng, Smon Rodrguez 1 Introducton In the last lecture we covered the ellpsod method and ts applcaton
More informationErrors for Linear Systems
Errors for Lnear Systems When we solve a lnear system Ax b we often do not know A and b exactly, but have only approxmatons  and ˆb avalable. Then the best thng we can do s to solve ˆx ˆb exactly whch
More informationDifferentiating Gaussian Processes
Dfferentatng Gaussan Processes Andrew McHutchon Aprl 17, 013 1 Frst Order Dervatve of the Posteror Mean The posteror mean of a GP s gven by, f = x, X KX, X 1 y x, X α 1 Only the x, X term depends on the
More informationAn Interactive Optimisation Tool for Allocation Problems
An Interactve Optmsaton ool for Allocaton Problems Fredr Bonäs, Joam Westerlund and apo Westerlund Process Desgn Laboratory, Faculty of echnology, Åbo Aadem Unversty, uru 20500, Fnland hs paper presents
More informationHidden Markov Models
Hdden Markov Models Namrata Vaswan, Iowa State Unversty Aprl 24, 204 Hdden Markov Model Defntons and Examples Defntons:. A hdden Markov model (HMM) refers to a set of hdden states X 0, X,..., X t,...,
More informationAmiri s Supply Chain Model. System Engineering b Department of Mathematics and Statistics c Odette School of Business
Amr s Supply Chan Model by S. Ashtab a,, R.J. Caron b E. Selvarajah c a Department of Industral Manufacturng System Engneerng b Department of Mathematcs Statstcs c Odette School of Busness Unversty of
More informationThe Minimum Universal Cost Flow in an Infeasible Flow Network
Journal of Scences, Islamc Republc of Iran 17(2): 175-180 (2006) Unversty of Tehran, ISSN 1016-1104 http://jscencesutacr The Mnmum Unversal Cost Flow n an Infeasble Flow Network H Saleh Fathabad * M Bagheran
More informationMEM 255 Introduction to Control Systems Review: Basics of Linear Algebra
MEM 255 Introducton to Control Systems Revew: Bascs of Lnear Algebra Harry G. Kwatny Department of Mechancal Engneerng & Mechancs Drexel Unversty Outlne Vectors Matrces MATLAB Advanced Topcs Vectors A
More informationChapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems
Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons
More information4DVAR, according to the name, is a four-dimensional variational method.
4D-Varatonal Data Assmlaton (4D-Var) 4DVAR, accordng to the name, s a four-dmensonal varatonal method. 4D-Var s actually a drect generalzaton of 3D-Var to handle observatons that are dstrbuted n tme. The
More informationLearning Theory: Lecture Notes
Learnng Theory: Lecture Notes Lecturer: Kamalka Chaudhur Scrbe: Qush Wang October 27, 2012 1 The Agnostc PAC Model Recall that one of the constrants of the PAC model s that the data dstrbuton has to be
More informationLecture 4: Universal Hash Functions/Streaming Cont d
CSE 5: Desgn and Analyss of Algorthms I Sprng 06 Lecture 4: Unversal Hash Functons/Streamng Cont d Lecturer: Shayan Oves Gharan Aprl 6th Scrbe: Jacob Schreber Dsclamer: These notes have not been subjected
More informationSociété de Calcul Mathématique SA
Socété de Calcul Mathématque SA Outls d'ade à la décson Tools for decson help Probablstc Studes: Normalzng the Hstograms Bernard Beauzamy December, 202 I. General constructon of the hstogram Any probablstc
More informationDifference Equations
Dfference Equatons c Jan Vrbk 1 Bascs Suppose a sequence of numbers, say a 0,a 1,a,a 3,... s defned by a certan general relatonshp between, say, three consecutve values of the sequence, e.g. a + +3a +1
More informationFormulas for the Determinant
page 224 224 CHAPTER 3 Determnants e t te t e 2t 38 A = e t 2te t e 2t e t te t 2e 2t 39 If 123 A = 345, 456 compute the matrx product A adj(a) What can you conclude about det(a)? For Problems 40 43, use
More informationExpected Value and Variance
MATH 38 Expected Value and Varance Dr. Neal, WKU We now shall dscuss how to fnd the average and standard devaton of a random varable X. Expected Value Defnton. The expected value (or average value, or
More informationDynamic Programming. Lecture 13 (5/31/2017)
Dynamc Programmng Lecture 13 (5/31/2017) - A Forest Thnnng Example - Projected yeld (m3/ha) at age 20 as functon of acton taken at age 10 Age 10 Begnnng Volume Resdual Ten-year Volume volume thnned volume
More informationLOW BIAS INTEGRATED PATH ESTIMATORS. James M. Calvin
Proceedngs of the 007 Wnter Smulaton Conference S G Henderson, B Bller, M-H Hseh, J Shortle, J D Tew, and R R Barton, eds LOW BIAS INTEGRATED PATH ESTIMATORS James M Calvn Department of Computer Scence
More informationGlobal Optimization of Truss. Structure Design INFORMS J. N. Hooker. Tallys Yunes. Slide 1
Slde 1 Global Optmzaton of Truss Structure Desgn J. N. Hooker Tallys Yunes INFORMS 2010 Truss Structure Desgn Select sze of each bar (possbly zero) to support the load whle mnmzng weght. Bar szes are dscrete.
More informationMaximal Margin Classifier
CS81B/Stat41B: Advanced Topcs n Learnng & Decson Makng Mamal Margn Classfer Lecturer: Mchael Jordan Scrbes: Jana van Greunen Corrected verson - /1/004 1 References/Recommended Readng 1.1 Webstes www.kernel-machnes.org
More informationNotes on Frequency Estimation in Data Streams
Notes on Frequency Estmaton n Data Streams In (one of) the data streamng model(s), the data s a sequence of arrvals a 1, a 2,..., a m of the form a j = (, v) where s the dentty of the tem and belongs to
More informationTime-Varying Systems and Computations Lecture 6
Tme-Varyng Systems and Computatons Lecture 6 Klaus Depold 14. Januar 2014 The Kalman Flter The Kalman estmaton flter attempts to estmate the actual state of an unknown dscrete dynamcal system, gven nosy
More informationHomework Notes Week 7
Homework Notes Week 7 Math 4 Sprng 4 #4 (a Complete the proof n example 5 that s an nner product (the Frobenus nner product on M n n (F In the example propertes (a and (d have already been verfed so we
More informationResource Allocation with a Budget Constraint for Computing Independent Tasks in the Cloud
Resource Allocaton wth a Budget Constrant for Computng Independent Tasks n the Cloud Wemng Sh and Bo Hong School of Electrcal and Computer Engneerng Georga Insttute of Technology, USA 2nd IEEE Internatonal
More informationADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING
1 ADVANCED ACHINE LEARNING ADVANCED ACHINE LEARNING Non-lnear regresson technques 2 ADVANCED ACHINE LEARNING Regresson: Prncple N ap N-dm. nput x to a contnuous output y. Learn a functon of the type: N
More information= = = (a) Use the MATLAB command rref to solve the system. (b) Let A be the coefficient matrix and B be the right-hand side of the system.
Chapter Matlab Exercses Chapter Matlab Exercses. Consder the lnear system of Example n Secton.. x x x y z y y z (a) Use the MATLAB command rref to solve the system. (b) Let A be the coeffcent matrx and
More informationApplied Stochastic Processes
STAT455/855 Fall 23 Appled Stochastc Processes Fnal Exam, Bref Solutons 1. (15 marks) (a) (7 marks) The dstrbuton of Y s gven by ( ) ( ) y 2 1 5 P (Y y) for y 2, 3,... The above follows because each of
More informationVARIATION OF CONSTANT SUM CONSTRAINT FOR INTEGER MODEL WITH NON UNIFORM VARIABLES
VARIATION OF CONSTANT SUM CONSTRAINT FOR INTEGER MODEL WITH NON UNIFORM VARIABLES BÂRZĂ, Slvu Faculty of Mathematcs-Informatcs Spru Haret Unversty barza_slvu@yahoo.com Abstract Ths paper wants to contnue
More informationQuantum Mechanics for Scientists and Engineers. David Miller
Quantum Mechancs for Scentsts and Engneers Davd Mller Types of lnear operators Types of lnear operators Blnear expanson of operators Blnear expanson of lnear operators We know that we can expand functons
More informationModule 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Verson ECE IIT, Kharagpur Lesson 6 Theory of Quantzaton Verson ECE IIT, Kharagpur Instructonal Objectves At the end of ths lesson, the students should be able to:
More informationWhich Separator? Spring 1
Whch Separator? 6.034 - Sprng 1 Whch Separator? Mamze the margn to closest ponts 6.034 - Sprng Whch Separator? Mamze the margn to closest ponts 6.034 - Sprng 3 Margn of a pont " # y (w $ + b) proportonal
More information1 Matrix representations of canonical matrices
1 Matrx representatons of canoncal matrces 2-d rotaton around the orgn: ( ) cos θ sn θ R 0 = sn θ cos θ 3-d rotaton around the x-axs: R x = 1 0 0 0 cos θ sn θ 0 sn θ cos θ 3-d rotaton around the y-axs:
More informationMin Cut, Fast Cut, Polynomial Identities
Randomzed Algorthms, Summer 016 Mn Cut, Fast Cut, Polynomal Identtes Instructor: Thomas Kesselhem and Kurt Mehlhorn 1 Mn Cuts n Graphs Lecture (5 pages) Throughout ths secton, G = (V, E) s a mult-graph.
More informationMean Field / Variational Approximations
Mean Feld / Varatonal Appromatons resented by Jose Nuñez 0/24/05 Outlne Introducton Mean Feld Appromaton Structured Mean Feld Weghted Mean Feld Varatonal Methods Introducton roblem: We have dstrbuton but
More informationLinear Feature Engineering 11
Lnear Feature Engneerng 11 2 Least-Squares 2.1 Smple least-squares Consder the followng dataset. We have a bunch of nputs x and correspondng outputs y. The partcular values n ths dataset are x y 0.23 0.19
More informationVQ widely used in coding speech, image, and video
at Scalar quantzers are specal cases of vector quantzers (VQ): they are constraned to look at one sample at a tme (memoryless) VQ does not have such constrant better RD perfomance expected Source codng
More informationC4B Machine Learning Answers II. = σ(z) (1 σ(z)) 1 1 e z. e z = σ(1 σ) (1 + e z )
C4B Machne Learnng Answers II.(a) Show that for the logstc sgmod functon dσ(z) dz = σ(z) ( σ(z)) A. Zsserman, Hlary Term 20 Start from the defnton of σ(z) Note that Then σ(z) = σ = dσ(z) dz = + e z e z
More informationReport on Image warping
Report on Image warpng Xuan Ne, Dec. 20, 2004 Ths document summarzed the algorthms of our mage warpng soluton for further study, and there s a detaled descrpton about the mplementaton of these algorthms.
More informationC/CS/Phy191 Problem Set 3 Solutions Out: Oct 1, 2008., where ( 00. ), so the overall state of the system is ) ( ( ( ( 00 ± 11 ), Φ ± = 1
C/CS/Phy9 Problem Set 3 Solutons Out: Oct, 8 Suppose you have two qubts n some arbtrary entangled state ψ You apply the teleportaton protocol to each of the qubts separately What s the resultng state obtaned
More informationA New Refinement of Jacobi Method for Solution of Linear System Equations AX=b
Int J Contemp Math Scences, Vol 3, 28, no 17, 819-827 A New Refnement of Jacob Method for Soluton of Lnear System Equatons AX=b F Naem Dafchah Department of Mathematcs, Faculty of Scences Unversty of Gulan,
More informationFeb 14: Spatial analysis of data fields
Feb 4: Spatal analyss of data felds Mappng rregularly sampled data onto a regular grd Many analyss technques for geophyscal data requre the data be located at regular ntervals n space and/or tme. hs s
More information1 Convex Optimization
Convex Optmzaton We wll consder convex optmzaton problems. Namely, mnmzaton problems where the objectve s convex (we assume no constrants for now). Such problems often arse n machne learnng. For example,
More information8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS
SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 493 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces you have studed thus far n the text are real vector spaces because the scalars
More informationLECTURE 9 CANONICAL CORRELATION ANALYSIS
LECURE 9 CANONICAL CORRELAION ANALYSIS Introducton he concept of canoncal correlaton arses when we want to quantfy the assocatons between two sets of varables. For example, suppose that the frst set of
More informationLecture 3 January 31, 2017
CS 224: Advanced Algorthms Sprng 207 Prof. Jelan Nelson Lecture 3 January 3, 207 Scrbe: Saketh Rama Overvew In the last lecture we covered Y-fast tres and Fuson Trees. In ths lecture we start our dscusson
More informationBezier curves. Michael S. Floater. August 25, These notes provide an introduction to Bezier curves. i=0
Bezer curves Mchael S. Floater August 25, 211 These notes provde an ntroducton to Bezer curves. 1 Bernsten polynomals Recall that a real polynomal of a real varable x R, wth degree n, s a functon of the
More information1 Generating functions, continued
Generatng functons, contnued. Generatng functons and parttons We can make use of generatng functons to answer some questons a bt more restrctve than we ve done so far: Queston : Fnd a generatng functon
More informationLecture 10 Support Vector Machines. Oct
Lecture 10 Support Vector Machnes Oct - 20-2008 Lnear Separators Whch of the lnear separators s optmal? Concept of Margn Recall that n Perceptron, we learned that the convergence rate of the Perceptron
More informationOn the Repeating Group Finding Problem
The 9th Workshop on Combnatoral Mathematcs and Computaton Theory On the Repeatng Group Fndng Problem Bo-Ren Kung, Wen-Hsen Chen, R.C.T Lee Graduate Insttute of Informaton Technology and Management Takmng
More information18.1 Introduction and Recap
CS787: Advanced Algorthms Scrbe: Pryananda Shenoy and Shjn Kong Lecturer: Shuch Chawla Topc: Streamng Algorthmscontnued) Date: 0/26/2007 We contnue talng about streamng algorthms n ths lecture, ncludng
More informationProblem Do any of the following determine homomorphisms from GL n (C) to GL n (C)?
Homework 8 solutons. Problem 16.1. Whch of the followng defne homomomorphsms from C\{0} to C\{0}? Answer. a) f 1 : z z Yes, f 1 s a homomorphsm. We have that z s the complex conjugate of z. If z 1,z 2
More informationSpectral Clustering. Shannon Quinn
Spectral Clusterng Shannon Qunn (wth thanks to Wllam Cohen of Carnege Mellon Unverst, and J. Leskovec, A. Raaraman, and J. Ullman of Stanford Unverst) Graph Parttonng Undrected graph B- parttonng task:
More information