The Problem: Mapping programs to architectures
|
|
- Austen Hensley
- 6 years ago
- Views:
Transcription
1 Complng for Parallelsm & Localty!Last tme! SSA and ts uses!today! Parallelsm and localty! Data dependences and loops CS553 Lecture Complng for Parallelsm & Localty 1 The Problem: Mappng programs to archtectures Goal: keep each core as busy as possble. Challenge: get the data to the core when t needs t From Modelng Parallel Computers as Memory Herarches by B. Alpern and L. Carter and J. Ferrante, From Sequoa: Programmng the Memory Herarchy by Fatahalan et al., CS553 Lecture Complng for Parallelsm & Localty 2
2 ! A(,)! A()! A() Example 1: Loop Permutaton for Improved Localty!Sample code: Assume Fortran s Column Maor Order array layout = 1,6 = 1,5 = A(,)+1 do = 1,5 = 1,6! A(,) = A(,) poor cache localty good cache localty CS553 Lecture Complng for Parallelsm & Localty Example 2: Parallelzaton!Can we parallelze the followng loops? = 1,100 = A() Yes = 1,100 = A(-1) No CS553 Lecture Complng for Parallelsm & Localty 4
3 ! s! s! s! A() Data Dependences!Recall! A data dependence defnes orderng relatonshp two between statements! In executng statements, data dependences must be respected to preserve correctness!example 1 a := 5; s 1 a := 5;!? 2 b := a + 1; s 3 a := 6; 3 a := 6; s 2 b := a + 1; CS553 Lecture Complng for Parallelsm & Localty 5 Data Dependences and Loops!How do we dentfy dependences n loops? = 1,5 = A(-1)+1 A(1) = A(0)+1!Smple vew! Imagne that all loops are fully unrolled! Examne data dependences as before!problems!!impractcal and often mpossble!!lose loop structure A(2) = A(1)+1 A(3) = A(2)+1 A(4) = A(3)+1 A(5) = A(4)+1 CS553 Lecture Complng for Parallelsm & Localty 6
4 ! A()! C()! A() Concepts needed for automatng loop transformatons!questons! How do we determne f a transformaton or parallelzaton s legal?! What abstracton do we use for loops?! How do we represent transformatons and parallelzaton?! How do we generate the transformed code?! How do we determne when a transformaton s gong to be benefcal?!today! Basc abstractons for loops and dependences and computng dependences!thursday! Abstractons for loop transformatons and determnng ther legalty! Code generaton after performng a loop transformaton CS553 Lecture Complng for Parallelsm & Localty 7 Dependences and Loops!Loop-ndependent dependences = 1,100 = B()+1 = A()*2 Dependences wthn the same loop teraton!loop-carred dependences = 1,100 = B()+1 C() = A(-1)*2 Dependences that cross loop teratons CS553 Lecture Complng for Parallelsm & Localty 8
5 !...! A(f(!...! Heurstcs! Use Dependence Testng n General!General code 1 = l 1,h 1 n = l n,h n 1,..., n )) =... A(g( 1,..., n ))!There exsts a dependence between teratons I=( 1,..., n ) and J=( 1,..., n ) when! f(i) = g(j)! (l 1,...l n ) < I,J < (h 1,...,h n )! I < J or J < I, where < s lexcographcally less CS553 Lecture Data Dependence Analyss 9 Algorthms for Solvng the Dependence Problem can say NO or MAYBE! GCD test (Baneree76,Towle76): determnes whether nteger soluton s possble, no bounds checkng! Baneree test (Baneree 79): checks real bounds! Independent-varables test (pg. 820): useful when nequaltes are not coupled! I-Test (Kong et al. 90): nteger soluton n real bounds! Lambda test (L et al. 90): all dmensons smultaneously! Delta test (Goff et al. 91): pattern matches for effcency! Power test (Wolfe et al. 92): extended GCD and Fourer Motzkn combnaton some form of Fourer-Motzkn elmnaton for ntegers, exponental worst-case! Parametrc Integer Programmng (Feautrer91)! Omega test (Pugh92) CS553 Lecture Data Dependence Analyss 10
6 ! A(a*+c Dependence Testng!Consder the followng code = 1,5 A(3*+2) = A(2*+1)+1!Queston! How do we determne whether one array reference depends on another across teratons of an teraton space? CS553 Lecture Data Dependence Analyss 11 Dependence Testng: Smple Case!Sample code = l,h 1 ) =... A(a*+c 2 )!Dependence?! a* 1 +c 1 = a* 2 +c 2, or! a* 1 a* 2 = c 2 -c 1! Soluton may exst f a dvdes c 2 -c 1 CS553 Lecture Data Dependence Analyss 12
7 ! A(a! A(4* GCD Test!Idea! Generalze test to lnear functons of terators/nducton varables!code = l,h = l,h 1 * + a 2 * + a 0 ) =... A(b 1 * + b 2 * + b 0 )...!Agan! a 1 * 1 - b 1 * 2 + a 2 * 1 b 2 * 2 = b 0 a 0! Soluton exsts f gcd(a 1,a 2,b 1,b 2 ) dvdes b 0 a 0 CS553 Lecture Data Dependence Analyss 13 Example!Code = l,h = l,h + 2* + 1) =... A(6* + 2* + 4)...!gcd(4,-6,2,-2) = 2!Does 2 dvde 4-1? CS553 Lecture Data Dependence Analyss 14
8 ! A(,) Baneree Test for (=L; <=U; ++) {! x[a0 + a1*] =...!... = x[b0 + b1*]! }! Does a0 + a1* = b0 + b1* for some real and? If so then (a1* - b1* ) = (b0 - a0)! Determne upper and lower bounds on (a1* - b1* ) for (=1; <=5; ++) { x[+5] = x[]; } upper bound = a1*max() - b1 * mn( ) = 4 lower bound = a1*mn() - b1*max( ) = -4 b_0 - a_0 = CS553 Lecture Data Dependence Analyss 15 Example 1: Loop Permutaton (reprse)!sample code = 1,6 = 1,5 = A(,)+1 do = 1,5 = 1,6! A(,) = A(,)+1!Why s ths legal?! No loop-carred dependences, so we can arbtrarly change order of teraton executon CS553 Lecture Complng for Parallelsm & Localty 16
9 ! A()! A()! A(,) Example 2: Parallelzaton (reprse)!why can t ths loop be parallelzed? = 1,100 = A(-1)+1!Why can ths loop be parallelzed? Loop carred dependence = 1,100 = A() No loop carred dependence, No soluton to dependence problem CS553 Lecture Complng for Parallelsm & Localty 17 Iteraton Spaces!Idea! Explctly represent the teratons of a loop nest!example = 1,6 = 1,5 = A(-1,-1)+1!Iteraton Space! A set of tuples that represents the teratons of a loop! Can vsualze the dependences n an teraton space Iteraton Space CS553 Lecture Complng for Parallelsm & Localty 18
10 ! A(,) Dstance Vectors!Idea! Concsely descrbe dependence relatonshps between teratons of an teraton space! For each dmenson of an teraton space, the dstance s the number of teratons between accesses to the same memory locaton!defnton! v = T - S!Example!do = 1,6 = 1,5! A(,) = A(-1,-2)+1! outer loop!dstance Vector: (1,2) nner loop CS553 Lecture Complng for Parallelsm & Localty 19 Dstance Vectors and Loop Transformatons!Idea! Any transformaton we perform on the loop must respect the dependences!example = 1,6 = 1,5 = A(-1,-2)+1!Can we permute the and loops? CS553 Lecture Complng for Parallelsm & Localty 20
11 ! A(,) Dstance Vectors and Loop Transformatons!Idea! Any transformaton we perform on the loop must respect the dependences!example = 1,5 = 1,6 = A(-1,-2)+1!Can we permute the and loops?! Yes CS553 Lecture Complng for Parallelsm & Localty 21 Dstance Vectors: Legalty!Defnton! A dependence vector, v, s lexcographcally nonnegatve when the leftmost entry n v s postve or all elements of v are zero Yes: (0,0,0), (0,1), (0,2,-2) No: (-1), (0,-2), (0,-1,1)! A dependence vector s legal when t s lexcographcally nonnegatve (assumng that ndces ncrease as we terate)!why are lexcographcally negatve dstance vectors llegal?!what are legal drecton vectors? CS553 Lecture Complng for Parallelsm & Localty 22
12 ! A(2*+2)! 2*! (yes, Data Dependence Termnology!We say statement s 2 depends on s 1! True (flow) dependence: s 1 wrtes memory that s 2 later reads! Ant-dependence: s 1 reads memory that s 2 later wrtes! Output dependences: s 1 wrtes memory that s 2 later wrtes! Input dependences: s 1 reads memory that s 2 later reads!notaton: s 1 " s 2! s 1 s called the source of the dependence! s 2 s called the snk or target! s 1 must be executed before s 2 CS553 Lecture Complng for Parallelsm & Localty 23 Example!Code 1 = l,h = A(2*-2)+1!Dependence? 1 2* 2 = -2 2 = -4 2 dvdes -4) 2!Knd of dependence?! Ant? 2 + d = 1 # d = -2!!Flow? 1 + d = 2 # d = 2 CS553 Lecture Data Dependence Analyss 24
13 ! A(,)! A(,) Example Sample code = 1,6 = 1,5 = A(-1,+1)+1!Knd of dependence: Flow!Dstance vector: (1,!1) CS553 Lecture Complng for Parallelsm & Localty 25 Exercse Sample code = 1,5 = 1,6 = A(-1,+1)+1!Knd of dependence: Ant!Dstance vector: (1, -1) CS553 Lecture Complng for Parallelsm & Localty 26
14 ! A()! A()! h[,0]! f[,0]! for! f[,]! EE[,]! h[,]! h[,]! p[,]! HH[,]! score[,] Example 2: Parallelzaton (reprse)!why can t ths loop be parallelzed? = 1,100 = A(-1) Dstance Vector: (1)!Why can ths loop be parallelzed? = 1,100 = A() Dstance Vector: (0) CS553 Lecture Complng for Parallelsm & Localty 27 Proten Strng Matchng Example!q = k_1!!r = k_2!!score[,] = 0 for the whole array!!for =1 to n1-1! = p[,0] = 0! = -q! =1 to n0-1! = max(f[,-1],h[,-1]-q)-r! = max(ee[-1,],hh[-1,],-q)-r! = p[,-1] + pam2[aa1[],aa0[]]! = max( max(0,ee[,]), max(f[,],h[,])! = HH[-1,]! = h[,]! = max(score[,-1],h[,])!! endfor!!endfor!!return score[n1-1,n0-1]! CS553 Lecture Complng for Parallelsm & Localty 28
15 ! A(,)! B(,)! A(,)! B(,) Loop-Carred Dependences!Defnton! A dependence D=(d 1,...d n ) s carred at loop level f d s the frst nonzero element of D!Example = 1,6 = 1,6 = B(-1,)+1 = A(,-1)*2!Dstance vectors:!loop-carred dependences (0,1) for accesses to A (1,0) for accesses to B! The loop carres dependence due to A! The loop carres dependence due to B CS553 Lecture Data Dependence Analyss 29 Parallelzaton!Idea! Each teraton of a loop may be executed n parallel f that loop carres no dependences!example (dfferent from last slde) = 1,5 = 1,6 = B(-1,-1)+1 = A(,-1)*2!Parallelze loop? Dstance Vectors: (1,0) for A (flow) (1,1) for B (flow) CS553 Lecture Data Dependence Analyss 30 Iteraton Space
16 ! t! C()!! A(,) Loop-Carred, Storage-Related Dependences!Problem! Loop-carred dependences nhbt parallelsm! Scalar references result n loop-carred dependences!example = 1,6 = A() + B() = t + 1/t!Can ths loop be parallelzed?!what knd of dependences are these? No. Ant dependences. Conventon for these sldes: Arrays start wth upper case letters, scalars do not CS553 Lecture Data Dependence Analyss 31 Drecton Vector!Defnton! A drecton vector serves the same purpose as a dstance vector when less precson s requred or avalable! Element of a drecton vector s <, >, or = based on whether the source of the dependence precedes, follows or s n the same teraton as the target n loop!example = 1,6 = 1,5 = A(-1,-1)+1!Drecton vector:!dstance vector: (<,<) (1,1) CS553 Lecture Data Dependence Analyss 32
17 ! T()! C()! Removng False Dependences wth Scalar Expanson!Idea! Elmnate false dependences by ntroducng extra storage!example = 1,6 = A() + B() = T() + 1/T()!Can ths loop be parallelzed?!dsadvantages? CS553 Lecture Data Dependence Analyss 33 Scalar Expanson Detals!Restrctons! The loop must be a countable loop.e. The loop trp count must be ndependent of the body of the loop! The expanded scalar must have no upward exposed uses n the loop do = 1,6 prnt(t) t = A() + B() C() = t + 1/t!! Nested loops may requre much more storage!! When the scalar s lve after the loop, we must move the correct array value nto the scalar CS553 Lecture Data Dependence Analyss 34
18 Concepts!Improve performance by...! mprovng data localty! parallelzng the computaton Data Dependence Testng! general formulaton of the problem! GCD test and Baneree test Data Dependences! teraton space! dstance vectors and drecton vectors! loop carred Transformaton legalty! must respect data dependences CS553 Lecture Data Dependence Analyss 35 Next Tme!Readng! Ch !Lecture! Abstractons for loop transformatons and checkng ther legalty! Code generaton after transformatons have been performed CS553 Lecture Complng for Parallelsm & Localty 36
Compiling for Parallelism & Locality. Example. Announcement Need to make up November 14th lecture. Last time Data dependences and loops
Complng for Parallelsm & Localty Announcement Need to make up November 14th lecture Last tme Data dependences and loops Today Fnsh data dependence analyss for loops CS553 Lecture Complng for Parallelsm
More informationCommon loop optimizations. Example to improve locality. Why Dependence Analysis. Data Dependence in Loops. Goal is to find best schedule:
15-745 Lecture 6 Data Dependence n Loops Copyrght Seth Goldsten, 2008 Based on sldes from Allen&Kennedy Lecture 6 15-745 2005-8 1 Common loop optmzatons Hostng of loop-nvarant computatons pre-compute before
More informationInner Product. Euclidean Space. Orthonormal Basis. Orthogonal
Inner Product Defnton 1 () A Eucldean space s a fnte-dmensonal vector space over the reals R, wth an nner product,. Defnton 2 (Inner Product) An nner product, on a real vector space X s a symmetrc, blnear,
More informationAnnouncements PA2 due Friday Midterm is Wednesday next week, in class, one week from today
Loop Transformations Announcements PA2 due Friday Midterm is Wednesday next week, in class, one week from today Today Recall stencil computations Intro to loop transformations Data dependencies between
More informationMEM 255 Introduction to Control Systems Review: Basics of Linear Algebra
MEM 255 Introducton to Control Systems Revew: Bascs of Lnear Algebra Harry G. Kwatny Department of Mechancal Engneerng & Mechancs Drexel Unversty Outlne Vectors Matrces MATLAB Advanced Topcs Vectors A
More information8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS
SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 493 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces you have studed thus far n the text are real vector spaces because the scalars
More informationFeb 14: Spatial analysis of data fields
Feb 4: Spatal analyss of data felds Mappng rregularly sampled data onto a regular grd Many analyss technques for geophyscal data requre the data be located at regular ntervals n space and/or tme. hs s
More informationStructure and Drive Paul A. Jensen Copyright July 20, 2003
Structure and Drve Paul A. Jensen Copyrght July 20, 2003 A system s made up of several operatons wth flow passng between them. The structure of the system descrbes the flow paths from nputs to outputs.
More informationPHYS 705: Classical Mechanics. Calculus of Variations II
1 PHYS 705: Classcal Mechancs Calculus of Varatons II 2 Calculus of Varatons: Generalzaton (no constrant yet) Suppose now that F depends on several dependent varables : We need to fnd such that has a statonary
More informationU.C. Berkeley CS294: Beyond Worst-Case Analysis Luca Trevisan September 5, 2017
U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 4s Luca Trevsan September 5, 07 Summary of Lecture 4 In whch we ntroduce semdefnte programmng and apply t to Max Cut. Semdefnte Programmng Recall that
More informationA 2D Bounded Linear Program (H,c) 2D Linear Programming
A 2D Bounded Lnear Program (H,c) h 3 v h 8 h 5 c h 4 h h 6 h 7 h 2 2D Lnear Programmng C s a polygonal regon, the ntersecton of n halfplanes. (H, c) s nfeasble, as C s empty. Feasble regon C s unbounded
More informationExample: (13320, 22140) =? Solution #1: The divisors of are 1, 2, 3, 4, 5, 6, 9, 10, 12, 15, 18, 20, 27, 30, 36, 41,
The greatest common dvsor of two ntegers a and b (not both zero) s the largest nteger whch s a common factor of both a and b. We denote ths number by gcd(a, b), or smply (a, b) when there s no confuson
More informationFeature Selection: Part 1
CSE 546: Machne Learnng Lecture 5 Feature Selecton: Part 1 Instructor: Sham Kakade 1 Regresson n the hgh dmensonal settng How do we learn when the number of features d s greater than the sample sze n?
More informationChapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems
Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons
More informationDesign and Analysis of Algorithms
Desgn and Analyss of Algorthms CSE 53 Lecture 4 Dynamc Programmng Junzhou Huang, Ph.D. Department of Computer Scence and Engneerng CSE53 Desgn and Analyss of Algorthms The General Dynamc Programmng Technque
More informationReport on Image warping
Report on Image warpng Xuan Ne, Dec. 20, 2004 Ths document summarzed the algorthms of our mage warpng soluton for further study, and there s a detaled descrpton about the mplementaton of these algorthms.
More informationThe Study of Teaching-learning-based Optimization Algorithm
Advanced Scence and Technology Letters Vol. (AST 06), pp.05- http://dx.do.org/0.57/astl.06. The Study of Teachng-learnng-based Optmzaton Algorthm u Sun, Yan fu, Lele Kong, Haolang Q,, Helongang Insttute
More informationWeek3, Chapter 4. Position and Displacement. Motion in Two Dimensions. Instantaneous Velocity. Average Velocity
Week3, Chapter 4 Moton n Two Dmensons Lecture Quz A partcle confned to moton along the x axs moves wth constant acceleraton from x =.0 m to x = 8.0 m durng a 1-s tme nterval. The velocty of the partcle
More informationLecture 11. minimize. c j x j. j=1. 1 x j 0 j. +, b R m + and c R n +
Topcs n Theoretcal Computer Scence May 4, 2015 Lecturer: Ola Svensson Lecture 11 Scrbes: Vncent Eggerlng, Smon Rodrguez 1 Introducton In the last lecture we covered the ellpsod method and ts applcaton
More informationn α j x j = 0 j=1 has a nontrivial solution. Here A is the n k matrix whose jth column is the vector for all t j=0
MODULE 2 Topcs: Lnear ndependence, bass and dmenson We have seen that f n a set of vectors one vector s a lnear combnaton of the remanng vectors n the set then the span of the set s unchanged f that vector
More informationUnit 5: Quadratic Equations & Functions
Date Perod Unt 5: Quadratc Equatons & Functons DAY TOPIC 1 Modelng Data wth Quadratc Functons Factorng Quadratc Epressons 3 Solvng Quadratc Equatons 4 Comple Numbers Smplfcaton, Addton/Subtracton & Multplcaton
More informationLecture 3. Ax x i a i. i i
18.409 The Behavor of Algorthms n Practce 2/14/2 Lecturer: Dan Spelman Lecture 3 Scrbe: Arvnd Sankar 1 Largest sngular value In order to bound the condton number, we need an upper bound on the largest
More informationCalculation of time complexity (3%)
Problem 1. (30%) Calculaton of tme complexty (3%) Gven n ctes, usng exhaust search to see every result takes O(n!). Calculaton of tme needed to solve the problem (2%) 40 ctes:40! dfferent tours 40 add
More informationProblem Set 9 Solutions
Desgn and Analyss of Algorthms May 4, 2015 Massachusetts Insttute of Technology 6.046J/18.410J Profs. Erk Demane, Srn Devadas, and Nancy Lynch Problem Set 9 Solutons Problem Set 9 Solutons Ths problem
More informationUnque Sets Orented Parallelzaton of Loops wth Non-unform Dependences Example : Example : Example : do =, do =, do =, do =, do =, do =, A( + ; ) = A( +
Unque Sets Orented Parallelzaton of Loops wth Non-unform Dependences Jaln Ju and Vpn Chaudhary Parallel and Dstrbuted Computng Laboratory, Wayne State Unversty, Detrot, MI, USA Emal: vpn@eng.wayne.edu
More informationp 1 c 2 + p 2 c 2 + p 3 c p m c 2
Where to put a faclty? Gven locatons p 1,..., p m n R n of m houses, want to choose a locaton c n R n for the fre staton. Want c to be as close as possble to all the house. We know how to measure dstance
More informationNumerical Heat and Mass Transfer
Master degree n Mechancal Engneerng Numercal Heat and Mass Transfer 06-Fnte-Dfference Method (One-dmensonal, steady state heat conducton) Fausto Arpno f.arpno@uncas.t Introducton Why we use models and
More information2.3 Nilpotent endomorphisms
s a block dagonal matrx, wth A Mat dm U (C) In fact, we can assume that B = B 1 B k, wth B an ordered bass of U, and that A = [f U ] B, where f U : U U s the restrcton of f to U 40 23 Nlpotent endomorphsms
More informationHomework 9 Solutions. 1. (Exercises from the book, 6 th edition, 6.6, 1-3.) Determine the number of distinct orderings of the letters given:
Homework 9 Solutons PROBLEM ONE 1 (Exercses from the book, th edton,, 1-) Determne the number of dstnct orderngs of the letters gven: (a) GUIDE Soluton: 5! (b) SCHOOL Soluton:! (c) SALESPERSONS Soluton:
More informationFundamental loop-current method using virtual voltage sources technique for special cases
Fundamental loop-current method usng vrtual voltage sources technque for specal cases George E. Chatzaraks, 1 Marna D. Tortorel 1 and Anastasos D. Tzolas 1 Electrcal and Electroncs Engneerng Departments,
More informationDynamic Programming. Preview. Dynamic Programming. Dynamic Programming. Dynamic Programming (Example: Fibonacci Sequence)
/24/27 Prevew Fbonacc Sequence Longest Common Subsequence Dynamc programmng s a method for solvng complex problems by breakng them down nto smpler sub-problems. It s applcable to problems exhbtng the propertes
More informationLecture 21: Numerical methods for pricing American type derivatives
Lecture 21: Numercal methods for prcng Amercan type dervatves Xaoguang Wang STAT 598W Aprl 10th, 2014 (STAT 598W) Lecture 21 1 / 26 Outlne 1 Fnte Dfference Method Explct Method Penalty Method (STAT 598W)
More informationThe Minimum Universal Cost Flow in an Infeasible Flow Network
Journal of Scences, Islamc Republc of Iran 17(2): 175-180 (2006) Unversty of Tehran, ISSN 1016-1104 http://jscencesutacr The Mnmum Unversal Cost Flow n an Infeasble Flow Network H Saleh Fathabad * M Bagheran
More informationDiscussion 11 Summary 11/20/2018
Dscusson 11 Summary 11/20/2018 1 Quz 8 1. Prove for any sets A, B that A = A B ff B A. Soluton: There are two drectons we need to prove: (a) A = A B B A, (b) B A A = A B. (a) Frst, we prove A = A B B A.
More informationVQ widely used in coding speech, image, and video
at Scalar quantzers are specal cases of vector quantzers (VQ): they are constraned to look at one sample at a tme (memoryless) VQ does not have such constrant better RD perfomance expected Source codng
More informationOn the Multicriteria Integer Network Flow Problem
BULGARIAN ACADEMY OF SCIENCES CYBERNETICS AND INFORMATION TECHNOLOGIES Volume 5, No 2 Sofa 2005 On the Multcrtera Integer Network Flow Problem Vassl Vasslev, Marana Nkolova, Maryana Vassleva Insttute of
More information8.6 The Complex Number System
8.6 The Complex Number System Earler n the chapter, we mentoned that we cannot have a negatve under a square root, snce the square of any postve or negatve number s always postve. In ths secton we want
More information18.1 Introduction and Recap
CS787: Advanced Algorthms Scrbe: Pryananda Shenoy and Shjn Kong Lecturer: Shuch Chawla Topc: Streamng Algorthmscontnued) Date: 0/26/2007 We contnue talng about streamng algorthms n ths lecture, ncludng
More informationa b a In case b 0, a being divisible by b is the same as to say that
Secton 6.2 Dvsblty among the ntegers An nteger a ε s dvsble by b ε f there s an nteger c ε such that a = bc. Note that s dvsble by any nteger b, snce = b. On the other hand, a s dvsble by only f a = :
More informationMMA and GCMMA two methods for nonlinear optimization
MMA and GCMMA two methods for nonlnear optmzaton Krster Svanberg Optmzaton and Systems Theory, KTH, Stockholm, Sweden. krlle@math.kth.se Ths note descrbes the algorthms used n the author s 2007 mplementatons
More informationRelaxation Methods for Iterative Solution to Linear Systems of Equations
Relaxaton Methods for Iteratve Soluton to Lnear Systems of Equatons Gerald Recktenwald Portland State Unversty Mechancal Engneerng Department gerry@pdx.edu Overvew Techncal topcs Basc Concepts Statonary
More informationLecture 10 Support Vector Machines II
Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the fake-test data; fxed
More informationImage classification. Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing i them?
Image classfcaton Gven te bag-of-features representatons of mages from dfferent classes ow do we learn a model for dstngusng tem? Classfers Learn a decson rule assgnng bag-offeatures representatons of
More informationTHEOREMS OF QUANTUM MECHANICS
THEOREMS OF QUANTUM MECHANICS In order to develop methods to treat many-electron systems (atoms & molecules), many of the theorems of quantum mechancs are useful. Useful Notaton The matrx element A mn
More informationMath 261 Exercise sheet 2
Math 261 Exercse sheet 2 http://staff.aub.edu.lb/~nm116/teachng/2017/math261/ndex.html Verson: September 25, 2017 Answers are due for Monday 25 September, 11AM. The use of calculators s allowed. Exercse
More informationVARIATION OF CONSTANT SUM CONSTRAINT FOR INTEGER MODEL WITH NON UNIFORM VARIABLES
VARIATION OF CONSTANT SUM CONSTRAINT FOR INTEGER MODEL WITH NON UNIFORM VARIABLES BÂRZĂ, Slvu Faculty of Mathematcs-Informatcs Spru Haret Unversty barza_slvu@yahoo.com Abstract Ths paper wants to contnue
More informationLecture 4: Universal Hash Functions/Streaming Cont d
CSE 5: Desgn and Analyss of Algorthms I Sprng 06 Lecture 4: Unversal Hash Functons/Streamng Cont d Lecturer: Shayan Oves Gharan Aprl 6th Scrbe: Jacob Schreber Dsclamer: These notes have not been subjected
More informationA Simple Inventory System
A Smple Inventory System Lawrence M. Leems and Stephen K. Park, Dscrete-Event Smulaton: A Frst Course, Prentce Hall, 2006 Hu Chen Computer Scence Vrgna State Unversty Petersburg, Vrgna February 8, 2017
More informationA New Refinement of Jacobi Method for Solution of Linear System Equations AX=b
Int J Contemp Math Scences, Vol 3, 28, no 17, 819-827 A New Refnement of Jacob Method for Soluton of Lnear System Equatons AX=b F Naem Dafchah Department of Mathematcs, Faculty of Scences Unversty of Gulan,
More informationLecture 3: Dual problems and Kernels
Lecture 3: Dual problems and Kernels C4B Machne Learnng Hlary 211 A. Zsserman Prmal and dual forms Lnear separablty revsted Feature mappng Kernels for SVMs Kernel trck requrements radal bass functons SVM
More informationEcon107 Applied Econometrics Topic 3: Classical Model (Studenmund, Chapter 4)
I. Classcal Assumptons Econ7 Appled Econometrcs Topc 3: Classcal Model (Studenmund, Chapter 4) We have defned OLS and studed some algebrac propertes of OLS. In ths topc we wll study statstcal propertes
More informationOutline and Reading. Dynamic Programming. Dynamic Programming revealed. Computing Fibonacci. The General Dynamic Programming Technique
Outlne and Readng Dynamc Programmng The General Technque ( 5.3.2) -1 Knapsac Problem ( 5.3.3) Matrx Chan-Product ( 5.3.1) Dynamc Programmng verson 1.4 1 Dynamc Programmng verson 1.4 2 Dynamc Programmng
More informationCHAPTER 14 GENERAL PERTURBATION THEORY
CHAPTER 4 GENERAL PERTURBATION THEORY 4 Introducton A partcle n orbt around a pont mass or a sphercally symmetrc mass dstrbuton s movng n a gravtatonal potental of the form GM / r In ths potental t moves
More informationSolution of Linear System of Equations and Matrix Inversion Gauss Seidel Iteration Method
Soluton of Lnear System of Equatons and Matr Inverson Gauss Sedel Iteraton Method It s another well-known teratve method for solvng a system of lnear equatons of the form a + a22 + + ann = b a2 + a222
More informationInductance Calculation for Conductors of Arbitrary Shape
CRYO/02/028 Aprl 5, 2002 Inductance Calculaton for Conductors of Arbtrary Shape L. Bottura Dstrbuton: Internal Summary In ths note we descrbe a method for the numercal calculaton of nductances among conductors
More information5 The Rational Canonical Form
5 The Ratonal Canoncal Form Here p s a monc rreducble factor of the mnmum polynomal m T and s not necessarly of degree one Let F p denote the feld constructed earler n the course, consstng of all matrces
More informationLINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity
LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 30 Multcollnearty Dr. Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur 2 Remedes for multcollnearty Varous technques have
More informationPerron Vectors of an Irreducible Nonnegative Interval Matrix
Perron Vectors of an Irreducble Nonnegatve Interval Matrx Jr Rohn August 4 2005 Abstract As s well known an rreducble nonnegatve matrx possesses a unquely determned Perron vector. As the man result of
More informationSupport Vector Machines CS434
Support Vector Machnes CS434 Lnear Separators Many lnear separators exst that perfectly classfy all tranng examples Whch of the lnear separators s the best? + + + + + + + + + Intuton of Margn Consder ponts
More informationTHE CHINESE REMAINDER THEOREM. We should thank the Chinese for their wonderful remainder theorem. Glenn Stevens
THE CHINESE REMAINDER THEOREM KEITH CONRAD We should thank the Chnese for ther wonderful remander theorem. Glenn Stevens 1. Introducton The Chnese remander theorem says we can unquely solve any par of
More informationHMMT February 2016 February 20, 2016
HMMT February 016 February 0, 016 Combnatorcs 1. For postve ntegers n, let S n be the set of ntegers x such that n dstnct lnes, no three concurrent, can dvde a plane nto x regons (for example, S = {3,
More informationMathematical Preparations
1 Introducton Mathematcal Preparatons The theory of relatvty was developed to explan experments whch studed the propagaton of electromagnetc radaton n movng coordnate systems. Wthn expermental error the
More informationLecture 17: Lee-Sidford Barrier
CSE 599: Interplay between Convex Optmzaton and Geometry Wnter 2018 Lecturer: Yn Tat Lee Lecture 17: Lee-Sdford Barrer Dsclamer: Please tell me any mstake you notced. In ths lecture, we talk about the
More informationDifference Equations
Dfference Equatons c Jan Vrbk 1 Bascs Suppose a sequence of numbers, say a 0,a 1,a,a 3,... s defned by a certan general relatonshp between, say, three consecutve values of the sequence, e.g. a + +3a +1
More informationEEL 6266 Power System Operation and Control. Chapter 3 Economic Dispatch Using Dynamic Programming
EEL 6266 Power System Operaton and Control Chapter 3 Economc Dspatch Usng Dynamc Programmng Pecewse Lnear Cost Functons Common practce many utltes prefer to represent ther generator cost functons as sngle-
More informationU.C. Berkeley CS294: Spectral Methods and Expanders Handout 8 Luca Trevisan February 17, 2016
U.C. Berkeley CS94: Spectral Methods and Expanders Handout 8 Luca Trevsan February 7, 06 Lecture 8: Spectral Algorthms Wrap-up In whch we talk about even more generalzatons of Cheeger s nequaltes, and
More informationMin Cut, Fast Cut, Polynomial Identities
Randomzed Algorthms, Summer 016 Mn Cut, Fast Cut, Polynomal Identtes Instructor: Thomas Kesselhem and Kurt Mehlhorn 1 Mn Cuts n Graphs Lecture (5 pages) Throughout ths secton, G = (V, E) s a mult-graph.
More informationRELIABILITY ASSESSMENT
CHAPTER Rsk Analyss n Engneerng and Economcs RELIABILITY ASSESSMENT A. J. Clark School of Engneerng Department of Cvl and Envronmental Engneerng 4a CHAPMAN HALL/CRC Rsk Analyss for Engneerng Department
More informationLecture Randomized Load Balancing strategies and their analysis. Probability concepts include, counting, the union bound, and Chernoff bounds.
U.C. Berkeley CS273: Parallel and Dstrbuted Theory Lecture 1 Professor Satsh Rao August 26, 2010 Lecturer: Satsh Rao Last revsed September 2, 2010 Lecture 1 1 Course Outlne We wll cover a samplng of the
More informationNP-Completeness : Proofs
NP-Completeness : Proofs Proof Methods A method to show a decson problem Π NP-complete s as follows. (1) Show Π NP. (2) Choose an NP-complete problem Π. (3) Show Π Π. A method to show an optmzaton problem
More informationComputing Correlated Equilibria in Multi-Player Games
Computng Correlated Equlbra n Mult-Player Games Chrstos H. Papadmtrou Presented by Zhanxang Huang December 7th, 2005 1 The Author Dr. Chrstos H. Papadmtrou CS professor at UC Berkley (taught at Harvard,
More informationSupport Vector Machines
CS 2750: Machne Learnng Support Vector Machnes Prof. Adrana Kovashka Unversty of Pttsburgh February 17, 2016 Announcement Homework 2 deadlne s now 2/29 We ll have covered everythng you need today or at
More informationCSci 6974 and ECSE 6966 Math. Tech. for Vision, Graphics and Robotics Lecture 21, April 17, 2006 Estimating A Plane Homography
CSc 6974 and ECSE 6966 Math. Tech. for Vson, Graphcs and Robotcs Lecture 21, Aprl 17, 2006 Estmatng A Plane Homography Overvew We contnue wth a dscusson of the major ssues, usng estmaton of plane projectve
More informationThe Second Anti-Mathima on Game Theory
The Second Ant-Mathma on Game Theory Ath. Kehagas December 1 2006 1 Introducton In ths note we wll examne the noton of game equlbrum for three types of games 1. 2-player 2-acton zero-sum games 2. 2-player
More informationVector Norms. Chapter 7 Iterative Techniques in Matrix Algebra. Cauchy-Bunyakovsky-Schwarz Inequality for Sums. Distances. Convergence.
Vector Norms Chapter 7 Iteratve Technques n Matrx Algebra Per-Olof Persson persson@berkeley.edu Department of Mathematcs Unversty of Calforna, Berkeley Math 128B Numercal Analyss Defnton A vector norm
More information3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X
Statstcs 1: Probablty Theory II 37 3 EPECTATION OF SEVERAL RANDOM VARIABLES As n Probablty Theory I, the nterest n most stuatons les not on the actual dstrbuton of a random vector, but rather on a number
More informationIntroduction to Algorithms
Introducton to Algorthms 6.046J/8.40J Lecture 7 Prof. Potr Indyk Data Structures Role of data structures: Encapsulate data Support certan operatons (e.g., INSERT, DELETE, SEARCH) Our focus: effcency of
More informationBasic Regular Expressions. Introduction. Introduction to Computability. Theory. Motivation. Lecture4: Regular Expressions
Introducton to Computablty Theory Lecture: egular Expressons Prof Amos Israel Motvaton If one wants to descrbe a regular language, La, she can use the a DFA, Dor an NFA N, such L ( D = La that that Ths
More informationCSE 252C: Computer Vision III
CSE 252C: Computer Vson III Lecturer: Serge Belonge Scrbe: Catherne Wah LECTURE 15 Kernel Machnes 15.1. Kernels We wll study two methods based on a specal knd of functon k(x, y) called a kernel: Kernel
More informationSimultaneous Optimization of Berth Allocation, Quay Crane Assignment and Quay Crane Scheduling Problems in Container Terminals
Smultaneous Optmzaton of Berth Allocaton, Quay Crane Assgnment and Quay Crane Schedulng Problems n Contaner Termnals Necat Aras, Yavuz Türkoğulları, Z. Caner Taşkın, Kuban Altınel Abstract In ths work,
More informationComputer-Aided Circuit Simulation and Verification. CSE245 Fall 2004 Professor:Chung-Kuan Cheng
Computer-Aded Crcut Smulaton and Verfcaton CSE245 Fall 24 Professor:Chung-Kuan Cheng Admnstraton Lectures: 5:pm ~ 6:2pm TTH HSS 252 Offce Hours: 4:pm ~ 4:45pm TTH APM 4256 Textbook Electronc Crcut and
More informationGrid Generation around a Cylinder by Complex Potential Functions
Research Journal of Appled Scences, Engneerng and Technolog 4(): 53-535, 0 ISSN: 040-7467 Mawell Scentfc Organzaton, 0 Submtted: December 0, 0 Accepted: Januar, 0 Publshed: June 0, 0 Grd Generaton around
More informationBernoulli Numbers and Polynomials
Bernoull Numbers and Polynomals T. Muthukumar tmk@tk.ac.n 17 Jun 2014 The sum of frst n natural numbers 1, 2, 3,..., n s n n(n + 1 S 1 (n := m = = n2 2 2 + n 2. Ths formula can be derved by notng that
More informationSection 8.3 Polar Form of Complex Numbers
80 Chapter 8 Secton 8 Polar Form of Complex Numbers From prevous classes, you may have encountered magnary numbers the square roots of negatve numbers and, more generally, complex numbers whch are the
More informationKernels in Support Vector Machines. Based on lectures of Martin Law, University of Michigan
Kernels n Support Vector Machnes Based on lectures of Martn Law, Unversty of Mchgan Non Lnear separable problems AND OR NOT() The XOR problem cannot be solved wth a perceptron. XOR Per Lug Martell - Systems
More informationCOMPLEX NUMBERS AND QUADRATIC EQUATIONS
COMPLEX NUMBERS AND QUADRATIC EQUATIONS INTRODUCTION We know that x 0 for all x R e the square of a real number (whether postve, negatve or ero) s non-negatve Hence the equatons x, x, x + 7 0 etc are not
More informationC/CS/Phy191 Problem Set 3 Solutions Out: Oct 1, 2008., where ( 00. ), so the overall state of the system is ) ( ( ( ( 00 ± 11 ), Φ ± = 1
C/CS/Phy9 Problem Set 3 Solutons Out: Oct, 8 Suppose you have two qubts n some arbtrary entangled state ψ You apply the teleportaton protocol to each of the qubts separately What s the resultng state obtaned
More informationPredictive Analytics : QM901.1x Prof U Dinesh Kumar, IIMB. All Rights Reserved, Indian Institute of Management Bangalore
Sesson Outlne Introducton to classfcaton problems and dscrete choce models. Introducton to Logstcs Regresson. Logstc functon and Logt functon. Maxmum Lkelhood Estmator (MLE) for estmaton of LR parameters.
More informationErrors for Linear Systems
Errors for Lnear Systems When we solve a lnear system Ax b we often do not know A and b exactly, but have only approxmatons  and ˆb avalable. Then the best thng we can do s to solve ˆx ˆb exactly whch
More informationLinear Approximation with Regularization and Moving Least Squares
Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...
More informationLecture Notes on Linear Regression
Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume
More information10-701/ Machine Learning, Fall 2005 Homework 3
10-701/15-781 Machne Learnng, Fall 2005 Homework 3 Out: 10/20/05 Due: begnnng of the class 11/01/05 Instructons Contact questons-10701@autonlaborg for queston Problem 1 Regresson and Cross-valdaton [40
More informationLinear Feature Engineering 11
Lnear Feature Engneerng 11 2 Least-Squares 2.1 Smple least-squares Consder the followng dataset. We have a bunch of nputs x and correspondng outputs y. The partcular values n ths dataset are x y 0.23 0.19
More informationOpen string operator quantization
Open strng operator quantzaton Requred readng: Zwebach -4 Suggested readng: Polchnsk 3 Green, Schwarz, & Wtten 3 upto eq 33 The lght-cone strng as a feld theory: Today we wll dscuss the quantzaton of an
More informationCS 331 DESIGN AND ANALYSIS OF ALGORITHMS DYNAMIC PROGRAMMING. Dr. Daisy Tang
CS DESIGN ND NLYSIS OF LGORITHMS DYNMIC PROGRMMING Dr. Dasy Tang Dynamc Programmng Idea: Problems can be dvded nto stages Soluton s a sequence o decsons and the decson at the current stage s based on the
More information= z 20 z n. (k 20) + 4 z k = 4
Problem Set #7 solutons 7.2.. (a Fnd the coeffcent of z k n (z + z 5 + z 6 + z 7 + 5, k 20. We use the known seres expanson ( n+l ( z l l z n below: (z + z 5 + z 6 + z 7 + 5 (z 5 ( + z + z 2 + z + 5 5
More informationIntroduction to Vapor/Liquid Equilibrium, part 2. Raoult s Law:
CE304, Sprng 2004 Lecture 4 Introducton to Vapor/Lqud Equlbrum, part 2 Raoult s Law: The smplest model that allows us do VLE calculatons s obtaned when we assume that the vapor phase s an deal gas, and
More informationThe Geometry of Logit and Probit
The Geometry of Logt and Probt Ths short note s meant as a supplement to Chapters and 3 of Spatal Models of Parlamentary Votng and the notaton and reference to fgures n the text below s to those two chapters.
More informationCompilers. Spring term. Alfonso Ortega: Enrique Alfonseca: Chapter 4: Syntactic analysis
Complers Sprng term Alfonso Ortega: alfonso.ortega@uam.es nrque Alfonseca: enrque.alfonseca@uam.es Chapter : Syntactc analyss. Introducton. Bottom-up Analyss Syntax Analyser Concepts It analyses the context-ndependent
More informationAn Interactive Optimisation Tool for Allocation Problems
An Interactve Optmsaton ool for Allocaton Problems Fredr Bonäs, Joam Westerlund and apo Westerlund Process Desgn Laboratory, Faculty of echnology, Åbo Aadem Unversty, uru 20500, Fnland hs paper presents
More information