Matrix Multiplication
|
|
|
- Corey Tiffany Porter
- 6 years ago
- Views:
Transcription
1 Matrix Multiplication
2 Matrix Multiplication Matrix multiplication. Given two n-by-n matrices A and B, compute C = AB. n c ij = a ik b kj k=1 c 11 c 12 c 1n c 21 c 22 c 2n c n1 c n2 c nn = a 11 a 12 a 1n a 21 a 22 a 2n a n1 a n2 a nn b 11 b 12 b 1n b 21 b 22 b 2n b n1 b n2 b nn Brute force. Θ(n 3 ) arithmetic operations. Fundamental question. Can we improve upon brute force? 2
3 Matrix Multiplication: Warmup Divide-and-conquer. Divide: partition A and B into ½n-by-½n blocks. Conquer: multiply 8 ½n-by-½n recursively. Combine: add appropriate products using 4 matrix additions. C 11 C 12 = A 11 A 12 B 11 B 12 C 21 C 22 A 21 A 22 B 21 B 22 ( ) + ( A 12 B 21 ) ( ) + ( A 12 B 22 ) ( ) + ( A 22 B 21 ) ( ) + ( A 22 B 22 ) C 11 = A 11 B 11 C 12 = A 11 B 12 C 21 = A 21 B 11 C 22 = A 21 B 12 ( ) T(n) = 8T n/2 + Θ(n2 ) recursive calls add, form submatrices T(n) = Θ(n 3 ) 3
4 Matrix Multiplication: Key Idea Key idea. multiply 2-by-2 block matrices with only multiplications. C 11 C 12 = C 21 C 22 A 11 A 12 B 11 B 12 A 21 A 22 B 21 B 22 P 1 = A 11 (B 12 B 22 ) P 2 = ( A 11 + A 12 ) B 22 C 11 = P 5 + P 4 P 2 + P 6 C 12 = P 1 + P 2 C 21 = P 3 + P 4 C 22 = P 5 + P 1 P 3 P P 3 = ( A 21 + A 22 ) B 11 P 4 = A 22 (B 21 B 11 ) P 5 = ( A 11 + A 22 ) (B 11 + B 22 ) P 6 = ( A 12 A 22 ) (B 21 + B 22 ) P = ( A 11 A 21 ) (B 11 + B 12 ) multiplications. 18 = additions (or subtractions). 4
5 Fast Matrix Multiplication Fast matrix multiplication. (Strassen, 1969) Divide: partition A and B into ½n-by-½n blocks. Compute: 14 ½n-by-½n matrices via 10 matrix additions. Conquer: multiply ½n-by-½n matrices recursively. Combine: products into 4 terms using 8 matrix additions. Analysis. Assume n is a power of 2. T(n) = # arithmetic operations. ( ) T(n) = T n /2 + Θ(n2 ) recursive calls add, subtract T(n) = Θ(n log 2 ) = O(n 2.81 ) 5
6 Fast Matrix Multiplication in Practice Implementation issues. Sparsity. Caching effects. Numerical stability. Odd matrix dimensions. Crossover to classical algorithm around n = 128. Common misperception: "Strassen is only a theoretical curiosity." Advanced Computation Group at Apple Computer reports 8x speedup on G4 Velocity Engine when n ~ 2,500. Range of instances where it's useful is a subject of controversy. Remark. Can "Strassenize" Ax=b, determinant, eigenvalues, and other matrix ops. 6
7 Θ(n log 2 ) = O(n 2.81 ) Θ(n log 3 21 ) = O(n 2. ) Θ(n log ) = O(n 2.80 )
8 Q. Multiply two 2-by-2 matrices with only scalar multiplications? Θ(n log 2 ) = O(n 2.81 ) Θ(n log 3 21 ) = O(n 2. ) Θ(n log ) = O(n 2.80 )
9 Q. Multiply two 2-by-2 matrices with only scalar multiplications? Θ(n log 2 ) = O(n 2.81 ) Θ(n log 3 21 ) = O(n 2. ) Θ(n log ) = O(n 2.80 )
10 Q. Multiply two 2-by-2 matrices with only scalar multiplications? Θ(n log 2 ) = O(n 2.81 ) Θ(n log 3 21 ) = O(n 2. ) Θ(n log ) = O(n 2.80 )
11 Q. Multiply two 2-by-2 matrices with only scalar multiplications? Θ(n log 2 ) = O(n 2.81 ) Θ(n log 3 21 ) = O(n 2. ) Θ(n log ) = O(n 2.80 )
12 Q. Multiply two 2-by-2 matrices with only scalar multiplications? Θ(n log 2 ) = O(n 2.81 ) A. Impossible. [Hopcroft and Kerr, 191] Θ(n log 3 21 ) = O(n 2. ) Θ(n log ) = O(n 2.80 )
13 Q. Multiply two 2-by-2 matrices with only scalar multiplications? Θ(n log 2 ) = O(n 2.81 ) A. Impossible. [Hopcroft and Kerr, 191] Θ(n log 3 21 ) = O(n 2. ) Θ(n log ) = O(n 2.80 )
14 Q. Multiply two 2-by-2 matrices with only scalar multiplications? Θ(n log 2 ) = O(n 2.81 ) A. Impossible. [Hopcroft and Kerr, 191] Q. Two 3-by-3 matrices with only 21 scalar multiplications? Θ(n log 3 21 ) = O(n 2. ) Θ(n log ) = O(n 2.80 )
15 Q. Multiply two 2-by-2 matrices with only scalar multiplications? Θ(n log 2 ) = O(n 2.81 ) A. Impossible. [Hopcroft and Kerr, 191] Q. Two 3-by-3 matrices with only 21 scalar multiplications? A. Also impossible. Θ(n log 3 21 ) = O(n 2. ) Θ(n log ) = O(n 2.80 )
16 Q. Multiply two 2-by-2 matrices with only scalar multiplications? Θ(n log 2 ) = O(n 2.81 ) A. Impossible. [Hopcroft and Kerr, 191] Q. Two 3-by-3 matrices with only 21 scalar multiplications? A. Also impossible. Θ(n log 3 21 ) = O(n 2. ) Θ(n log ) = O(n 2.80 )
17 Q. Multiply two 2-by-2 matrices with only scalar multiplications? Θ(n log 2 ) = O(n 2.81 ) A. Impossible. [Hopcroft and Kerr, 191] Q. Two 3-by-3 matrices with only 21 scalar multiplications? A. Also impossible. Q. Two 0-by-0 matrices with only 143,640 scalar multiplications? Θ(n log 3 21 ) = O(n 2. ) Θ(n log ) = O(n 2.80 )
18 Q. Multiply two 2-by-2 matrices with only scalar multiplications? Θ(n log 2 ) = O(n 2.81 ) A. Impossible. [Hopcroft and Kerr, 191] Q. Two 3-by-3 matrices with only 21 scalar multiplications? A. Also impossible. Q. Two 0-by-0 matrices with only 143,640 scalar multiplications? A. Yes! [Pan, 1980] Θ(n log 3 21 ) = O(n 2. ) Θ(n log ) = O(n 2.80 )
19 Q. Multiply two 2-by-2 matrices with only scalar multiplications? Θ(n log 2 ) = O(n 2.81 ) A. Impossible. [Hopcroft and Kerr, 191] Q. Two 3-by-3 matrices with only 21 scalar multiplications? A. Also impossible. Q. Two 0-by-0 matrices with only 143,640 scalar multiplications? A. Yes! [Pan, 1980] Θ(n log 3 21 ) = O(n 2. ) Θ(n log ) = O(n 2.80 )
20 Q. Multiply two 2-by-2 matrices with only scalar multiplications? Θ(n log 2 ) = O(n 2.81 ) A. Impossible. [Hopcroft and Kerr, 191] Q. Two 3-by-3 matrices with only 21 scalar multiplications? A. Also impossible. Q. Two 0-by-0 matrices with only 143,640 scalar multiplications? A. Yes! [Pan, 1980] Θ(n log 3 21 ) = O(n 2. ) Decimal wars. Θ(n log ) = O(n 2.80 )
21 Q. Multiply two 2-by-2 matrices with only scalar multiplications? Θ(n log 2 ) = O(n 2.81 ) A. Impossible. [Hopcroft and Kerr, 191] Q. Two 3-by-3 matrices with only 21 scalar multiplications? A. Also impossible. Q. Two 0-by-0 matrices with only 143,640 scalar multiplications? A. Yes! [Pan, 1980] Θ(n log 3 21 ) = O(n 2. ) Decimal wars. December, 199: O(n ). Θ(n log ) = O(n 2.80 )
22 Q. Multiply two 2-by-2 matrices with only scalar multiplications? Θ(n log 2 ) = O(n 2.81 ) A. Impossible. [Hopcroft and Kerr, 191] Q. Two 3-by-3 matrices with only 21 scalar multiplications? A. Also impossible. Q. Two 0-by-0 matrices with only 143,640 scalar multiplications? A. Yes! [Pan, 1980] Θ(n log 3 21 ) = O(n 2. ) Decimal wars. December, 199: O(n ). January, 1980: O(n ). Θ(n log ) = O(n 2.80 )
23 Best known. O(n 2.36 ) [Coppersmith-Winograd, 198.] Conjecture. O(n 2+ε ) for any ε > 0. Caveat. Theoretical improvements to Strassen are progressively less practical. 8
! Break up problem into several parts. ! Solve each part recursively. ! Combine solutions to sub-problems into overall solution.
Divide-and-Conquer Chapter 5 Divide and Conquer Divide-and-conquer.! Break up problem into several parts.! Solve each part recursively.! Combine solutions to sub-problems into overall solution. Most common
Copyright 2000, Kevin Wayne 1
Divide-and-Conquer Chapter 5 Divide and Conquer Divide-and-conquer. Break up problem into several parts. Solve each part recursively. Combine solutions to sub-problems into overall solution. Most common
Chapter 5. Divide and Conquer CLRS 4.3. Slides by Kevin Wayne. Copyright 2005 Pearson-Addison Wesley. All rights reserved.
Chapter 5 Divide and Conquer CLRS 4.3 Slides by Kevin Wayne. Copyright 2005 Pearson-Addison Wesley. All rights reserved. 1 Divide-and-Conquer Divide-and-conquer. Break up problem into several parts. Solve
How to Multiply. 5.5 Integer Multiplication. Complex Multiplication. Integer Arithmetic. Complex multiplication. (a + bi) (c + di) = x + yi.
How to ultiply Slides by Kevin Wayne. Copyright 5 Pearson-Addison Wesley. All rights reserved. integers, matrices, and polynomials Complex ultiplication Complex multiplication. a + bi) c + di) = x + yi.
Chapter 5. Divide and Conquer CLRS 4.3. Slides by Kevin Wayne. Copyright 2005 Pearson-Addison Wesley. All rights reserved.
Chapter 5 Divide and Conquer CLRS 4.3 Slides by Kevin Wayne. Copyright 25 Pearson-Addison Wesley. All rights reserved. Divide-and-Conquer Divide-and-conquer. Break up problem into several parts. Solve
DIVIDE AND CONQUER II
DIVIDE AND CONQUER II master theorem integer multiplication matrix multiplication convolution and FFT Lecture slides by Kevin Wayne Copyright 2005 Pearson-Addison Wesley http://www.cs.princeton.edu/~wayne/kleinberg-tardos
Matrix Multiplication. Data Structures and Algorithms Andrei Bulatov
Matrix Multiplicatio Data Structures ad Algorithms Adrei Bulatov Algorithms Matrix Multiplicatio 7- Matrix Multiplicatio Matrix multiplicatio. Give two -by- matrices A ad B, compute A B. k kj ik ij b a
The Divide-and-Conquer Design Paradigm
CS473- Algorithms I Lecture 4 The Divide-and-Conquer Design Paradigm CS473 Lecture 4 1 The Divide-and-Conquer Design Paradigm 1. Divide the problem (instance) into subproblems. 2. Conquer the subproblems
Introduction to Algorithms 6.046J/18.401J/SMA5503
Introduction to Algorithms 6.046J/8.40J/SMA5503 Lecture 3 Prof. Piotr Indyk The divide-and-conquer design paradigm. Divide the problem (instance) into subproblems. 2. Conquer the subproblems by solving
Algorithms and Data Structures Strassen s Algorithm. ADS (2017/18) Lecture 4 slide 1
Algorithms and Data Structures Strassen s Algorithm ADS (2017/18) Lecture 4 slide 1 Tutorials Start in week (week 3) Tutorial allocations are linked from the course webpage http://www.inf.ed.ac.uk/teaching/courses/ads/
Chapter 5. Divide and Conquer. CS 350 Winter 2018
Chapter 5 Divide and Conquer CS 350 Winter 2018 1 Divide-and-Conquer Divide-and-conquer. Break up problem into several parts. Solve each part recursively. Combine solutions to sub-problems into overall
Copyright 2000, Kevin Wayne 1
//8 Fast Integer Division Too (!) Schönhage Strassen algorithm CS 8: Algorithm Design and Analysis Integer division. Given two n-bit (or less) integers s and t, compute quotient q = s / t and remainder
CS 580: Algorithm Design and Analysis
CS 580: Algorithm Design and Analysis Jeremiah Blocki Purdue University Spring 208 Announcement: Homework 3 due February 5 th at :59PM Final Exam (Tentative): Thursday, May 3 @ 8AM (PHYS 203) Recap: Divide
Tutorials. Algorithms and Data Structures Strassen s Algorithm. The Master Theorem for solving recurrences. The Master Theorem (cont d)
DS 2018/19 Lecture 4 slide 3 DS 2018/19 Lecture 4 slide 4 Tutorials lgorithms and Data Structures Strassen s lgorithm Start in week week 3 Tutorial allocations are linked from the course webpage http://www.inf.ed.ac.uk/teaching/courses/ads/
Divide and Conquer. Andreas Klappenecker. [based on slides by Prof. Welch]
![Divide and Conquer. Andreas Klappenecker. [based on slides by Prof. Welch]](/thumbs/84/89083836.jpg "Divide and Conquer. Andreas Klappenecker. [based on slides by Prof. Welch]")
Divide and Conquer Andreas Klappenecker [based on slides by Prof. Welch] Divide and Conquer Paradigm An important general technique for designing algorithms: divide problem into subproblems recursively
CS60020: Foundations of Algorithm Design and Machine Learning. Sourangshu Bhattacharya
CS60020: Foundations of Algorithm Design and Machine Learning Sourangshu Bhattacharya Matrix multiplication September 14, 2005 L2.27 Standard algorithm for i 1 to n do for j 1 ton do c ij 0 for k 1 to
The Master Theorem for solving recurrences. Algorithms and Data Structures Strassen s Algorithm. Tutorials. The Master Theorem (cont d)
The Master Theorem for solving recurrences lgorithms and Data Structures Strassen s lgorithm 23rd September, 2014 Theorem Let n 0 N, k N 0 and a, b R with a > 0 and b > 1, and let T : N R satisfy the following
Complexity of Matrix Multiplication and Bilinear Problems
Complexity of Matrix Multiplication and Bilinear Problems François Le Gall Graduate School of Informatics Kyoto University ADFOCS17 - Lecture 3 24 August 2017 Overview of the Lectures Fundamental techniques
Divide-and-conquer algorithm
Divide-and-conquer algorithm IDEA: n n matrix = 2 2 matrix of (n/2) (n/2) submatrices: r=ae+bg s=af+bh t =ce+dh u=cf+dg r t s u = a c e g September 15, 2004 Introduction to Algorithms L3.31 b d C = A B
Chapter 4 Divide-and-Conquer
Chapter 4 Divide-and-Conquer 1 About this lecture (1) Recall the divide-and-conquer paradigm, which we used for merge sort: Divide the problem into a number of subproblems that are smaller instances of
CMPSCI611: Three Divide-and-Conquer Examples Lecture 2
CMPSCI611: Three Divide-and-Conquer Examples Lecture 2 Last lecture we presented and analyzed Mergesort, a simple divide-and-conquer algorithm. We then stated and proved the Master Theorem, which gives
Divide and Conquer: Polynomial Multiplication Version of October 1 / 7, 24201
Divide and Conquer: Polynomial Multiplication Version of October 7, 2014 Divide and Conquer: Polynomial Multiplication Version of October 1 / 7, 24201 Outline Outline: Introduction The polynomial multiplication
CMPS 2200 Fall Divide-and-Conquer. Carola Wenk. Slides courtesy of Charles Leiserson with changes and additions by Carola Wenk
CMPS 2200 Fall 2017 Divide-and-Conquer Carola Wenk Slides courtesy of Charles Leiserson with changes and additions by Carola Wenk 1 The divide-and-conquer design paradigm 1. Divide the problem (instance)
DIVIDE AND CONQUER II
DIVIDE AND CONQUER II master theorem integer multiplication matrix multiplication convolution and FFT DIVIDE AND CONQUER II master theorem integer multiplication matrix multiplication convolution and FFT
CS 4424 Matrix multiplication
CS 4424 Matrix multiplication 1 Reminder: matrix multiplication Matrix-matrix product. Starting from a 1,1 a 1,n A =.. and B = a n,1 a n,n b 1,1 b 1,n.., b n,1 b n,n we get AB by multiplying A by all columns
CS 577 Introduction to Algorithms: Strassen s Algorithm and the Master Theorem
CS 577 Introduction to Algorithms: Jin-Yi Cai University of Wisconsin Madison In the last class, we described InsertionSort and showed that its worst-case running time is Θ(n 2 ). Check Figure 2.2 for
Class Note #14. In this class, we studied an algorithm for integer multiplication, which. 2 ) to θ(n
Class Note #14 Date: 03/01/2006 [Overall Information] In this class, we studied an algorithm for integer multiplication, which improved the running time from θ(n 2 ) to θ(n 1.59 ). We then used some of
Fast Matrix Multiplication Over GF3
Fast Matrix Multiplication Over GF3 Matthew Lambert Advised by Dr. B. David Saunders & Dr. Stephen Siegel February 13, 2015 Matrix Multiplication Outline Fast Matrix Multiplication Outline Fast Matrix
Matrices: 2.1 Operations with Matrices
Goals In this chapter and section we study matrix operations: Define matrix addition Define multiplication of matrix by a scalar, to be called scalar multiplication. Define multiplication of two matrices,
CPS 616 DIVIDE-AND-CONQUER 6-1
CPS 616 DIVIDE-AND-CONQUER 6-1 DIVIDE-AND-CONQUER Approach 1. Divide instance of problem into two or more smaller instances 2. Solve smaller instances recursively 3. Obtain solution to original (larger)
Fast Matrix Product Algorithms: From Theory To Practice
Introduction and Definitions The τ-theorem Pan s aggregation tables and the τ-theorem Software Implementation Conclusion Fast Matrix Product Algorithms: From Theory To Practice Thomas Sibut-Pinote Inria,
Chapter 2. Divide-and-conquer. 2.1 Strassen s algorithm
Chapter 2 Divide-and-conquer This chapter revisits the divide-and-conquer paradigms and explains how to solve recurrences, in particular, with the use of the master theorem. We first illustrate the concept
Group-theoretic approach to Fast Matrix Multiplication
Jenya Kirshtein 1 Group-theoretic approach to Fast Matrix Multiplication Jenya Kirshtein Department of Mathematics University of Denver Jenya Kirshtein 2 References 1. H. Cohn, C. Umans. Group-theoretic
Ma/CS 6b Class 12: Graphs and Matrices
Ma/CS 6b Class 2: Graphs and Matrices 3 3 v 5 v 4 v By Adam Sheffer Non-simple Graphs In this class we allow graphs to be nonsimple. We allow parallel edges, but not loops. Incidence Matrix Consider a
DIVIDE AND CONQUER II
DIVIDE AND CONQUER II DIVIDE AND CONQUER II master theorem integer multiplication matrix multiplication convolution and FFT master theorem integer multiplication matrix multiplication convolution and FFT
Chapter 1: Systems of linear equations and matrices. Section 1.1: Introduction to systems of linear equations
Chapter 1: Systems of linear equations and matrices Section 1.1: Introduction to systems of linear equations Definition: A linear equation in n variables can be expressed in the form a 1 x 1 + a 2 x 2
Kartsuba s Algorithm and Linear Time Selection
CS 374: Algorithms & Models of Computation, Fall 2015 Kartsuba s Algorithm and Linear Time Selection Lecture 09 September 22, 2015 Chandra & Manoj (UIUC) CS374 1 Fall 2015 1 / 32 Part I Fast Multiplication
Divide and Conquer. Chapter Overview. 1.2 Divide and Conquer
Chapter 1 Divide and Conquer 1.1 Overview This chapter introduces Divide and Conquer, which is a technique for designing recursive algorithms that are (sometimes) asymptotically efficient. It also presents
Numerical Linear Algebra
Numerical Linear Algebra By: David McQuilling; Jesus Caban Deng Li Jan.,31,006 CS51 Solving Linear Equations u + v = 8 4u + 9v = 1 A x b 4 9 u v = 8 1 Gaussian Elimination Start with the matrix representation
An introduction to parallel algorithms
An introduction to parallel algorithms Knut Mørken Department of Informatics Centre of Mathematics for Applications University of Oslo Winter School on Parallel Computing Geilo January 20 25, 2008 1/26
How to find good starting tensors for matrix multiplication
How to find good starting tensors for matrix multiplication Markus Bläser Saarland University Matrix multiplication z,... z,n..... z n,... z n,n = x,... x,n..... x n,... x n,n y,... y,n..... y n,... y
Divide-and-Conquer. a technique for designing algorithms
Divide-and-Conquer a technique for designing algorithms decomposing instance to be solved into subinstances of the same problem solving each subinstance combining subsolutions to obtain the solution to
Computational complexity and some Graph Theory
Graph Theory Lars Hellström January 22, 2014 Contents of todays lecture An important concern when choosing the algorithm to use for something (after basic requirements such as correctness and stability)
Design and Analysis of Algorithms
Design and Analysis of Algorithms CSE 5311 Lecture 5 Divide and Conquer: Fast Fourier Transform Junzhou Huang, Ph.D. Department of Computer Science and Engineering CSE5311 Design and Analysis of Algorithms
Fast reversion of formal power series
Fast reversion of formal power series Fredrik Johansson LFANT, INRIA Bordeaux RAIM, 2016-06-29, Banyuls-sur-mer 1 / 30 Reversion of power series F = exp(x) 1 = x + x 2 2! + x 3 3! + x 4 G = log(1 + x)
William Stallings Copyright 2010
A PPENDIX E B ASIC C ONCEPTS FROM L INEAR A LGEBRA William Stallings Copyright 2010 E.1 OPERATIONS ON VECTORS AND MATRICES...2 Arithmetic...2 Determinants...4 Inverse of a Matrix...5 E.2 LINEAR ALGEBRA
MATRICES. a m,1 a m,n A =
MATRICES Matrices are rectangular arrays of real or complex numbers With them, we define arithmetic operations that are generalizations of those for real and complex numbers The general form a matrix of
Harvard CS 121 and CSCI E-207 Lecture 12: General Context-Free Recognition
Harvard CS 121 and CSCI E-207 Lecture 12: General Context-Free Recognition Salil Vadhan October 11, 2012 Reading: Sipser, Section 2.3 and Section 2.1 (material on Chomsky Normal Form). Pumping Lemma for
CS/COE 1501 cs.pitt.edu/~bill/1501/ Integer Multiplication
CS/COE 1501 cs.pitt.edu/~bill/1501/ Integer Multiplication Integer multiplication Say we have 5 baskets with 8 apples in each How do we determine how many apples we have? Count them all? That would take
Matrix Arithmetic. j=1
An m n matrix is an array A = Matrix Arithmetic a 11 a 12 a 1n a 21 a 22 a 2n a m1 a m2 a mn of real numbers a ij An m n matrix has m rows and n columns a ij is the entry in the i-th row and j-th column
Lesson 12.7: Sequences and Series
Lesson 12.7: Sequences and Series May 30 7:11 AM Sequences Definition: A sequence is a set of numbers in a specific order. 2, 5, 8,. is an example of a sequence. Note: A sequence may have either a finite
Lecture 9: Submatrices and Some Special Matrices
Lecture 9: Submatrices and Some Special Matrices Winfried Just Department of Mathematics, Ohio University February 2, 2018 Submatrices A submatrix B of a given matrix A is any matrix that can be obtained
IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 1
IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 1 Automatic Reproduction of a Genius Algorithm: Strassen s Algorithm Revisited by Genetic Search Seunghyun Oh and Byung-Ro Moon, Member, IEEE Abstract In 1968,
CSE 421 Algorithms: Divide and Conquer
CSE 42 Algorithms: Divide and Conquer Larry Ruzzo Thanks to Richard Anderson, Paul Beame, Kevin Wayne for some slides Outline: General Idea algorithm design paradigms: divide and conquer Review of Merge
Divide & Conquer. CS 320, Fall Dr. Geri Georg, Instructor CS320 Div&Conq 1
Divide & Conquer CS 320, Fall 2017 Dr. Geri Georg, Instructor georg@colostate.edu CS320 Div&Conq 1 Strategy 1. Divide the problem up into equal sized sub problems 2. Solve the sub problems recursively
CS475: Linear Equations Gaussian Elimination LU Decomposition Wim Bohm Colorado State University
CS475: Linear Equations Gaussian Elimination LU Decomposition Wim Bohm Colorado State University Except as otherwise noted, the content of this presentation is licensed under the Creative Commons Attribution
Divide and Conquer algorithms
Divide and Conquer algorithms Another general method for constructing algorithms is given by the Divide and Conquer strategy. We assume that we have a problem with input that can be split into parts in
Phys 201. Matrices and Determinants
Phys 201 Matrices and Determinants 1 1.1 Matrices 1.2 Operations of matrices 1.3 Types of matrices 1.4 Properties of matrices 1.5 Determinants 1.6 Inverse of a 3 3 matrix 2 1.1 Matrices A 2 3 7 =! " 1
Fast reversion of power series
Fast reversion of power series Fredrik Johansson November 2011 Overview Fast power series arithmetic Fast composition and reversion (Brent and Kung, 1978) A new algorithm for reversion Implementation results
Fast Matrix Multiplication*
mathematics of computation, volume 28, number 125, January, 1974 Triangular Factorization and Inversion by Fast Matrix Multiplication* By James R. Bunch and John E. Hopcroft Abstract. The fast matrix multiplication
Powers of Tensors and Fast Matrix Multiplication
Powers of Tensors and Fast Matrix Multiplication François Le Gall Department of Computer Science Graduate School of Information Science and Technology The University of Tokyo Simons Institute, 12 November
Introduction to Matrices
214 Analysis and Design of Feedback Control Systems Introduction to Matrices Derek Rowell October 2002 Modern system dynamics is based upon a matrix representation of the dynamic equations governing the
A Review of Matrix Analysis
Matrix Notation Part Matrix Operations Matrices are simply rectangular arrays of quantities Each quantity in the array is called an element of the matrix and an element can be either a numerical value
Fast Convolution; Strassen s Method
Fast Convolution; Strassen s Method 1 Fast Convolution reduction to subquadratic time polynomial evaluation at complex roots of unity interpolation via evaluation at complex roots of unity 2 The Master
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 1 x 2. x n 8 (4) 3 4 2
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS SYSTEMS OF EQUATIONS AND MATRICES Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a
STAT 309: MATHEMATICAL COMPUTATIONS I FALL 2018 LECTURE 13
STAT 309: MATHEMATICAL COMPUTATIONS I FALL 208 LECTURE 3 need for pivoting we saw that under proper circumstances, we can write A LU where 0 0 0 u u 2 u n l 2 0 0 0 u 22 u 2n L l 3 l 32, U 0 0 0 l n l
Algorithms Design & Analysis. Dynamic Programming
Algorithms Design & Analysis Dynamic Programming Recap Divide-and-conquer design paradigm Today s topics Dynamic programming Design paradigm Assembly-line scheduling Matrix-chain multiplication Elements
The System of Linear Equations. Direct Methods. Xiaozhou Li.
1/16 The Direct Methods xiaozhouli@uestc.edu.cn http://xiaozhouli.com School of Mathematical Sciences University of Electronic Science and Technology of China Chengdu, China Does the LU factorization always
feb abhi shelat Matrix, FFT
L7 feb 11 2016 abhi shelat Matrix, FFT userid: = Using the standard method, how many multiplications does it take to multiply two NxN matrices? cos( /4) = cos( /2) = sin( /4) = sin( /2) = Mergesort Karatsuba
Solving Recurrences. Lecture 23 CS2110 Fall 2011
Solving Recurrences Lecture 23 CS2110 Fall 2011 1 Announcements Makeup Prelim 2 Monday 11/21 7:30-9pm Upson 5130 Please do not discuss the prelim with your classmates! Quiz 4 next Tuesday in class Topics:
10.34: Numerical Methods Applied to Chemical Engineering. Lecture 2: More basics of linear algebra Matrix norms, Condition number
10.34: Numerical Methods Applied to Chemical Engineering Lecture 2: More basics of linear algebra Matrix norms, Condition number 1 Recap Numerical error Operations Properties Scalars, vectors, and matrices
Divide-and-conquer: Order Statistics. Curs: Fall 2017
Divide-and-conquer: Order Statistics Curs: Fall 2017 The divide-and-conquer strategy. 1. Break the problem into smaller subproblems, 2. recursively solve each problem, 3. appropriately combine their answers.
RECAP How to find a maximum matching?
RECAP How to find a maximum matching? First characterize maximum matchings A maximal matching cannot be enlarged by adding another edge. A maximum matching of G is one of maximum size. Example. Maximum
A Divide-and-Conquer Algorithm for Functions of Triangular Matrices
A Divide-and-Conquer Algorithm for Functions of Triangular Matrices Ç. K. Koç Electrical & Computer Engineering Oregon State University Corvallis, Oregon 97331 Technical Report, June 1996 Abstract We propose
Vectors and matrices: matrices (Version 2) This is a very brief summary of my lecture notes.
Vectors and matrices: matrices (Version 2) This is a very brief summary of my lecture notes Matrices and linear equations A matrix is an m-by-n array of numbers A = a 11 a 12 a 13 a 1n a 21 a 22 a 23 a
LU Factorization. Marco Chiarandini. DM559 Linear and Integer Programming. Department of Mathematics & Computer Science University of Southern Denmark
DM559 Linear and Integer Programming LU Factorization Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark [Based on slides by Lieven Vandenberghe, UCLA] Outline
The Fast Fourier Transform: A Brief Overview. with Applications. Petros Kondylis. Petros Kondylis. December 4, 2014
December 4, 2014 Timeline Researcher Date Length of Sequence Application CF Gauss 1805 Any Composite Integer Interpolation of orbits of celestial bodies F Carlini 1828 12 Harmonic Analysis of Barometric
Fractions. Review R.7. Dr. Doug Ensley. January 7, Dr. Doug Ensley Review R.7
Review R.7 Dr. Doug Ensley January 7, 2015 Equivalence of fractions As long as c 0, a b = a c b c Equivalence of fractions As long as c 0, a b = a c b c Examples True or False? 10 18 = 2 5 2 9 = 5 9 10
DO NOT USE WITHOUT PERMISSION
PROGRESSION FOR DEVELOPING ALGEBRA UNDERSTANDING THROUGH GENERALIZING ARITHMETIC ACROSS GRADES 3-7: This curricular progression is intended to develop algebra understanding through generalizing arithmetic.
ENGR-1100 Introduction to Engineering Analysis. Lecture 21. Lecture outline
ENGR-1100 Introduction to Engineering Analysis Lecture 21 Lecture outline Procedure (algorithm) for finding the inverse of invertible matrix. Investigate the system of linear equation and invertibility
Strassen-like algorithms for symmetric tensor contractions
Strassen-like algorithms for symmetric tensor contractions Edgar Solomonik Theory Seminar University of Illinois at Urbana-Champaign September 18, 2017 1 / 28 Fast symmetric tensor contractions Outline
ENGR-1100 Introduction to Engineering Analysis. Lecture 21
ENGR-1100 Introduction to Engineering Analysis Lecture 21 Lecture outline Procedure (algorithm) for finding the inverse of invertible matrix. Investigate the system of linear equation and invertibility
THE COMPLEXITY OF THE QUATERNION PROD- UCT*
1 THE COMPLEXITY OF THE QUATERNION PROD- UCT* Thomas D. Howell Jean-Claude Lafon 1 ** TR 75-245 June 1975 2 Department of Computer Science, Cornell University, Ithaca, N.Y. * This research was supported
I. Approaches to bounding the exponent of matrix multiplication
I. Approaches to bounding the exponent of matrix multiplication Chris Umans Caltech Based on joint work with Noga Alon, Henry Cohn, Bobby Kleinberg, Amir Shpilka, Balazs Szegedy Modern Applications of
On the Exponent of the All Pairs Shortest Path Problem
On the Exponent of the All Pairs Shortest Path Problem Noga Alon Department of Mathematics Sackler Faculty of Exact Sciences Tel Aviv University Zvi Galil Department of Computer Science Sackler Faculty
Elementary maths for GMT
Elementary maths for GMT Linear Algebra Part 2: Matrices, Elimination and Determinant m n matrices The system of m linear equations in n variables x 1, x 2,, x n a 11 x 1 + a 12 x 2 + + a 1n x n = b 1
Asymptotic Analysis and Recurrences
Appendix A Asymptotic Analysis and Recurrences A.1 Overview We discuss the notion of asymptotic analysis and introduce O, Ω, Θ, and o notation. We then turn to the topic of recurrences, discussing several
Sequences. 1. Number sequences. 2. Arithmetic sequences. Consider the illustrated pattern of circles:
Sequences 1. Number sequences Consider the illustrated pattern of circles: The first layer has just one blue ball. The second layer has three pink balls. The third layer has five black balls. The fourth
Divide and Conquer Algorithms
Divide and Conquer Algorithms Introduction There exist many problems that can be solved using a divide-and-conquer algorithm. A divide-andconquer algorithm A follows these general guidelines. Divide Algorithm
10. Linear Systems of ODEs, Matrix multiplication, superposition principle (parts of sections )
c Dr. Igor Zelenko, Fall 2017 1 10. Linear Systems of ODEs, Matrix multiplication, superposition principle (parts of sections 7.2-7.4) 1. When each of the functions F 1, F 2,..., F n in right-hand side
Chapter 2. Recurrence Relations. Divide and Conquer. Divide and Conquer Strategy. Another Example: Merge Sort. Merge Sort Example. Merge Sort Example
Recurrence Relations Chapter 2 Divide and Conquer Equation or an inequality that describes a function by its values on smaller inputs. Recurrence relations arise when we analyze the running time of iterative
Linear Algebra Tutorial for Math3315/CSE3365 Daniel R. Reynolds
Linear Algebra Tutorial for Math3315/CSE3365 Daniel R. Reynolds These notes are meant to provide a brief introduction to the topics from Linear Algebra that will be useful in Math3315/CSE3365, Introduction
I&C 6N. Computational Linear Algebra
I&C 6N Computational Linear Algebra 1 Lecture 1: Scalars and Vectors What is a scalar? Computer representation of a scalar Scalar Equality Scalar Operations Addition and Multiplication What is a vector?
The divide-and-conquer strategy solves a problem by: 1. Breaking it into subproblems that are themselves smaller instances of the same type of problem
Chapter 2. Divide-and-conquer algorithms The divide-and-conquer strategy solves a problem by: 1. Breaking it into subproblems that are themselves smaller instances of the same type of problem. 2. Recursively
Dynamic Programming: Matrix chain multiplication (CLRS 15.2)
Dynamic Programming: Matrix chain multiplication (CLRS.) The problem Given a sequence of matrices A, A, A,..., A n, find the best way (using the minimal number of multiplications) to compute their product.
Topic 17. Analysis of Algorithms
Topic 17 Analysis of Algorithms Analysis of Algorithms- Review Efficiency of an algorithm can be measured in terms of : Time complexity: a measure of the amount of time required to execute an algorithm
1 Multiply Eq. E i by λ 0: (λe i ) (E i ) 2 Multiply Eq. E j by λ and add to Eq. E i : (E i + λe j ) (E i )
Direct Methods for Linear Systems Chapter Direct Methods for Solving Linear Systems Per-Olof Persson persson@berkeleyedu Department of Mathematics University of California, Berkeley Math 18A Numerical
Algebra 2 Matrices. Multiple Choice Identify the choice that best completes the statement or answers the question. 1. Find.
Algebra 2 Matrices Review Multiple Choice Identify the choice that best completes the statement or answers the question. 1. Find. Evaluate the determinant of the matrix. 2. 3. A matrix contains 48 elements.
Kevin James. MTHSC 3110 Section 2.1 Matrix Operations
MTHSC 3110 Section 2.1 Matrix Operations Notation Let A be an m n matrix, that is, m rows and n columns. We ll refer to the entries of A by their row and column indices. The entry in the i th row and j
Matrices. Chapter Definitions and Notations
Chapter 3 Matrices 3. Definitions and Notations Matrices are yet another mathematical object. Learning about matrices means learning what they are, how they are represented, the types of operations which