On the mathematical background of Google PageRank algorithm

Save this PDF as:

Size: px
Start display at page:

Transcription

1 Working Paper Series Department of Economics University of Verona On the mathematical background of Google PageRank algorithm Alberto Peretti, Alberto Roveda WP Number: 25 December 2014 ISSN: (paper), (online)

2 On the mathematical background of Google PageRank algorithm Alberto Peretti Alberto Roveda Department of Economics, University of Verona Verona, Italy December, 2014 Abstract The PageRank algorithm, the kernel of the method used by Google Search to give us the answer of a search we are asking in the web, contains a lot of mathematics. Maybe one could say that graphs and Markov chains theories are in the background, while the crucial steps are in a linear algebra context, as the eigenvalues of a matrix are involved. In this working paper we deal with all the mathematics we need to explain how the PageRank method works. Keywords. Graph theory, Linear mappings, Eigenvalues, Markov chains. AMS Classification. 15A18, 05C, 60J10 JEL Classification. C65 1

4 The first three sections are devoted to these three topics. In the fourth section we present the power method, an iterative method to compute the dominant eigenvalue and eigenvector, that is a fundamental part of the PageRank algorithm, and the original algorithm itself. 1 Some elements from graph theory Definition 1 A graph is a couple G = (V,E), where V = {v 1,v 2,...,v n } is a finite set of elements (the nodes of the graph) and E = {e 1,e 2,...,e m } V V is a subset of couples of nodes (the edges of the graph). If E is a subset of ordered couples of nodes, the graph G is an oriented graph (or direct graph). In the picture below on the left we have a possible representation of a graph G with V = {v 1,v 2,v 3,v 4 } and E = {(v 1,v 2 ),(v 1,v 3 ),(v 1,v 4 ),(v 2,v 3 ),(v 3,v 4 )}. On the right there is a representation of a direct graph G with V = {v 1,v 2,v 3,v 4 } and E = {(v 1,v 2 ),(v 1,v 3 ),(v 2,v 1 ),(v 2,v 3 ),(v 3,v 4 ),(v 4,v 1 )}. v 1 v 4 v 1 v 4 Non-direct graph Direct graph Figure 1: Graphs As we will be concentrated in the following on direct graphs mainly, let s now recall some classical definitions for this kind of graphs. Definition 2 If G = (V,E) is a direct graph, a direct path in G is a sequence of nodes (v 0,v 1,...,v k ) such that (v i,v i+1 ) E for each i = 0,1,...,k 1. We say that k is the length of the path. If none of the nodes, with the possible exception of v 0 and v k, appears in the sequence twice, the path is simple. If v 0 = v k the path is closed. A path that is simple and closed is called a cycle. For the definition of the concept of connectivity in a direct graph it may be important to consider a path regardless of direction of the edges. Definition 3 In the same context of the previous Definition, a non-direct path in G is a sequence of nodes (v 0,v 1,...,v k ) such that either (v i,v i+1 ) or (v i+1,v i ) belongs to E for each i = 0,1,...,k 1. Definition 4 A direct graph is connected if for every pair of nodes (v i,v ) in V there exists a non direct path that oins them. A direct graph is strongly connected if for every pair of nodes (v i,v ) in V there exists a direct path that oins them. 2 The algebraic background In the PageRank algorithm important aspects are related to the dominant eigenvalue of the Google matrix and its associated eigenvector. We are going to recall some topics in linear algebra that are important in the sequel. 3

5 v 1 v 4 v 1 v 4 Connected graph Strongly connected graph Figure 2: Connectivity 2.1 The adacency matrix First of all, there are important ways to represent a graph. We may for instance put all the relevant information about the graph in a matrix. Suppose G = (V,E) is a direct graph. Definition 5 We call the adacency matrix A of G the n n square matrix, where n = V, such that { 1 if (vi,v a i = ) E 0 if (v i,v ) / E. Example. Here is an example of a direct graph and its adacency matrix. v 1 v 6 v 5 v The graph The adacency matrix Figure 3: Graphs and matrices There can be many matrices associated to a graph. Often it is much easier to operate on a matrix that on a geometrical representation of the graph. 2.2 Permutation matrices Definition 6 An n n matrix P whose rows are obtained by any permutation of the rows of the identity matrix is called a permutation matrix. Example. An example of permutation matrix is P = The matrix P has been obtained with the permutation (3 1 2) of the rows of the identity matrix or equivalently with the permutation (2 3 1) of its columns. And the two permutations are the inverse of each other. 1 Regarding the effect of the multiplication by a permutation matrix, let s consider a simple example: a b c d e f = b c a a b c g h i e f d and d e f = a b c. g h i h i g g h i d e f 1 Think that if the first row goes to the k-th position, and then p k1 = 1, then the k-th column goes to the first position. 4

6 The effect of a right multiplication of a 3 3 matrix A by the permutation P is the permutation (2 3 1) of the columns of A. The effect of a left multiplication of A by P is the permutation (3 1 2) of the rows of A. Remark. In general, since the rows of an n n permutation matrix P are the fundamental vectors in R n, the result of PP T is the identity matrix and then the inverse of P is its transpose. In other words P is an orthogonal matrix. The reason is in what we have ust seen: if P has a certain permutation of rows, then P T has that permutation on its columns. In this way, looking at PP T as the right permutation of P by P T, we permute the columns of P with the permutation of its rows. But the two are inverse and then we obtain the identity matrix. In a similar way if we look at PP T as the left permutation of P T by P. Definition 7 Given any n n square matrix A, the result A = PAP T, where P is a permutation matrix, is called a symmetric permutation of A. Remark. By considering the effect of right and left multiplication by a permutation matrix, the effect of PAP T is a certain permutation of the rows followed by the same permutaion of the columns. We may remark also that from the associative property of the matrix multiplication the result is the same if we permute the columns first and the rows secondly or the rows first and the columns at a second time. Example. To see the algebraic effects on the matrix and the corresponding effects on the graph, let s make an example on the adacency matrix of a simple graph. Consider the direct graph G in Figure 4 together with its adacency matrix A. v 1 v A = Figure 4: Example on permutations Let s consider also the permutation matrix P = , obtained by the permutation ( ) of the rows of the identity 4 4 matrix. We have PAP T = The corresponding graph is the following v 1 v 4 5

7 It is easy to see that the effect of the symmetric permutation on the adacency matrix of a graph is ust to rename the nodes of the graph. On the graph there has been the permutation ( ) of the nodes, in the sense that: the new node 1 is the old node 3, the new node 2 is the old 1, the new node 3 is the old 4 and the new node 4 is the old 2. It is worthwhile to remark that the only left multiplication of the adacency matrix by a permutation P is not interesting and is not related to a permutation of the nodes in the graph. For example in our case PA = = In Figure 5 we can see the effects of the non symmetric permutation on the original graph. It is evident that in the result some important topological properties of the original graph are destroyed. v 1 v 4 v 1 v 4 Figure 5: A non simmetric permutation on a graph 2.3 Eigenvalues and eigenvectors Fundamental steps of the PageRank algorithm are related to eigenvalues. We recall that Definition 8 Given an n n real or complex matrix A, a real or complex number λ and a real or complex non null vector v are respectively an eigenvalue and an associated eigenvector of A if Av = λv. In a similar way, a real or complex non null vector w is a left eigenvector associated to λ if w T A = λw T. We recall that the eigenvalues of A are the roots of P(λ) = det(a λi), the characteristic polynomial of A. The multiplicity of λ as a root of the characteristic polynomial is the so called algebraic multiplicity m a (λ) of λ. In the sequel we shall be interested essentially on the real case, but it is important to keep in mind that for a polynomial, even with real coefficients, we can guarantee the existence ust of complex roots. The fundamental theorem of algebra states that every n 1 degree single-variable polynomial with complex coefficients has at least one complex root. The theorem may also be stated as follows: every n degree single-variable polynomial with complex coefficients has, counted with multiplicity, exactly n roots. The equivalence of the two statements can be proved through the use of successive polynomial division. This includes polynomials with real coefficients. There are important properties of the matrix A that bring to the existence of real eigenvalues (at least one or all). A fundamental result is that symmetric matrices have all the eigenvalues that are real. Our interest is on real eigenvalues λ R and associated real eigenvectors v R n. We shall indicate with: 6

8 V λ the eigenspace of the eigenvalue λ, that is the subspace of R n spanned by the eigenvectors associated to λ. Sp(A) the spectrum of the matrix A, that is the set of the eigenvalues of A. ρ(a) the spectral radius of A, that is ρ(a) = max 1 i n λ i. Sometimes we call spectral circle the circle in the complex plane with center at the origin and radius equal to the spectral radius. Obviously all the eigenvalues belong to the spectral circle. Remark. If λ is an eigenvalue of A, its associated eigenspace V λ is the set of solutions of the linear homogeneous system (A I)v = 0. The dimension of V λ as a subspace of R n is the so called geometric multiplicity m g (λ) of λ. For every eigenvalue λ we have that m g (λ) m a (λ). Definition 9 If A is an n n symmetric matrix and x is a non null vector in R n, the Rayleigh quotient R(A,x) is defined as It can be shown that R(A,x) = xt Ax x T x. x T Ax min x R n x T x = λ min that is the smallest eigenvalue of A. The minimum is attained at x = v min, where v min is an eigenvector associated to λ min. Similarly x T Ax max x R n x T x = λ max, the largest eigenvalue of A. The maximum is attained at x = v max, where v max is an eigenvector associated to λ max. The Rayleigh quotient is used in eigenvalue algorithms to obtain an eigenvalue approximation from an eigenvector approximation. We are going to see this shortly in the power method algorithm. 2.4 The dominant eigenvalue Fundamental results hold regarding the leading (or dominant) eigenvalue, that is the eigenvalue with maximum module (absolute value). The most famous among these results is probably the Perron-Frobenius theorem, that has an important role in the PageRang algorithm. Let us first give some definitions about matrices. Definition 10 A matrix A is positive if all its elements are positive, that is a i > 0 per every i,. A matrix A is non-negative if all its elements are non-negative, that is a i 0 per every i,. The Perron-Frobenius theorem essentially says that a real square matrix with positive elements has a unique largest real eigenvalue. Moreover, an associated eigenvector can be chosen to have positive components. A similar statement holds for certain classes of non-negative matrices. The first results are due to Perron, regarding positive matrices. A generalization was obtained later by Frobenius and deals with non-negative matrices. The theorem has important applications to probability theory (ergodicity of Markov chains), to economic theory (Leontiev s input-output model) and many other fields. 7

9 Theorem 1 (Perron-Frobenius for positive matrices) Let A be an n n positive matrix. Then the following statements hold. (i) There is an eigenvalue r (the so called Perron-Frobenius eigenvalue), that is real and positive, and for any other eigenvalue λ we have λ < r. Then r = ρ(a). (ii) r is a simple eigenvalue, that is a simple root of the characteristic polynomial. In other words its algebraic multiplicity is one (m a (r) = 1). As a consequence the eigenspace V r associated with r is one-dimensional. (iii) There exists a positive eigenvector v associated with r. Respectively, there exists a positive left eigenvector w. (iv) There are no other positive eigenvectors of A, except (positive) multiples of v (respectively, left eigenvectors except w). ( ) k A (v) lim = vw T, where the right and left eigenvectors are normalized, so that w T v = 1. Moreover, the k + r matrix vw T is the proection onto the eigenspace V r, the so called Perron proection. (vi) The Perron-Frobenius eigenvalue r satisfies the inequalities min i a i r max i a i. Example. Let s consider the positive matrix With the help of a scientific software 2 we get: A = >A=[1,2,3;4,5,6;7,8,9] A = >spec(a) ans = D-15 -->[R,diagevals] = spec(a) diagevals = D-15 R = At the first prompt the command A=[...] ust defines the matrix A. At the second prompt the spec(a) command gives the eigenvalues of A, namely λ 1 = , λ 2 = , λ 3 = We have been using Scilab for the numerical computations. 3 λ 3 is actually zero, as the matrix is singular. 8

10 At the third prompt again the spec(a) command, in a more complete form, gives the diagonal matrix similar to A (the eigenvalues are in the main diagonal) and a second matrix where the columns are normalized eigenvectors, respectively associated to the corresponding eigenvalues in the first matrix. Clearly λ 1 is the Perron-Frobenius eigenvalue r; for the other eigenvalues the inequalities λ 2 < r and λ 3 < r hold. The first column v 1 of the last matrix is a generator for the eigenspace V λ1. We can see that the eigenvector v 1 is negative and this confirms the thesis of Perron-Frobenius theorem, that is satisfied of course by v 1. Note also that the other two normalized eigenvectors v 2 and v 3 both have positive and negative components at the same time and then they do not have positive multiples. We may get the left eigenvectors with the same commands above, applied to the transpose matrix of A (A in Scilab). Here is Scilab: -->[R,diagevals] = spec(a ) diagevals = D-16 R = The eigenvalues are of course the same (the last is zero). The opposite of first column of the last matrix is the positive left eigenvector of the Perron-Frobenius eigenvalue. To check point 5 of the thesis we may compute some powers of A k /r k for increasing values of k. Some attempts show that already with k = 5 we are near to converge. Here is Scilab: -->A^5/ ^5 ans = get 4 We may define the vectors v and w as the positive right and left eigenvectors, compute the product vw T and we -->v = [ , , ] v = >norm(v) ans = 1. -->w = [ , , ] w = >norm(w) ans = 1. -->v *w ans = In Scilab you see actually v T w because the vectors are defined as row vectors. 9

11 It s interesting that the two matrices are not exactly the same. The elements are different more or less for a 5 percent. The reason is the condition on the normalization. The two eigenvectors must be normalized in order to ensure w T v = 1. As they are norm one vectors, we may ust divide one of the two by w T v. In Scilab we have -->v*w ans = >v = v/(v*w ) v = >v*w ans = 1. -->v *w ans = Now the two matrices are exactly the same. To conclude the example, it s immediate to verify point 6 of the thesis, as min i a i = 6 and max i a i = 24. Frobenius extended Perron s results to the case of non-negative matrices, with both important similarities and differences. It is easy to obtain for this kind of matrices the existence of a non-negative eigenvalue r, with a corresponding non-negative eigenvector, such that r λ for every other eigenvalue λ. The problem is to guarantee the uniqueness. In fact there may be eigenvalues with the same absolute value. Consider for example the matrix ( ) 0 1, 1 0 that is non-negative and has 1 and 1 as eigenvalues. Moreover non-negative matrices may have the maximal eigenvalue that is not strictly positive and is not a simple root of the characteristic polynomial. Take for example the matrix ( ) 0 1, 0 0 whose only eigenvalue is 0 with algebraic multiplicity 2. The main properties of the maximum eigenvalue seem to fail with no hope. Frobenius idea was to consider a restriction on the class of non-negative matrices, in order to bring the generalization to its maximum extent. Anyway, as we shall see, some properties of the positive matrices Perron eigenvalue are lost. The matrices for which Frobenius was able to extend Perron results are the so called irreducible matrices. They may be defined by means of one of a few equivalent properties, some purely algebraic, some other related to graph theory. We report some of these properties. Let A be an n n square matrix. A is irreducible if any one of the following equivalent properties holds. The first is a geometric property related to the linear transformation on R n associated to the matrix A. Let s make a remark first. We call a coordinate subspace any linear subspace of R n spanned by a proper non-trivial subset of the R n canonical basis {e 1,e 2,...,e n }. Property 1 A does not have invariant coordinate subspaces. 10

12 More explicitly, for every coordinate subspace its image through the linear transformation associated to the matrix A is not the subspace itself or a subset of it. Examples. The matrix ( ) is not irreducible, as in R 2 the coordinate subspace spanned by e 1 is invariant. In fact ( )( ) ( ) 1 1 x1 x1 = Conversely the matrix ( ) is irreducible. As the only coordinate subspaces in R 2 are either the span of e 1 or the span of e 2, it is easy to be convinced that none of the two is invariant. To make things clearer on how the invariance must be intended the further following example is worthwhile. With the matrix the images of the coordinate subspaces are and A = A x 1 0 = 0 x 1, A 0 x 2 = 0 0, A 0 0 = x 1 A x 1 x 2 = 0 x 1, A x 1 0 = 0 x 1, A 0 x 2 = x 1 x 3 The matrix is not irreducible as the are three cases in which the image is a subspace of the coordinate subspace. Property 2 A cannot be transformed into block upper triangular form by a permutation matrix P, that is ( ) PAP 1 B11 B cannot be of the form 12 0 B 22 where B 11 and B 22 are square matrices of order greater or equal to 1. x 1 x 3 x 3 Examples. The first matrix of the previous example is clearly already block upper triangular. The second matrix of the previous example cannot be put in block upper triangular form because the only nontrivial permutation matrix in R 2 is the matrix itself and 0 1 ( ) 3 ( ) 1 0 = In R 3 the matrix is not irreducible. In fact, taking the permutation matrix P = we have and the result is block upper triangular PAP T = =

13 Properties 1 and 2 are characterizations of an irreducible general matrix. For non-negative matrices other properties can be proved. The following is a typical algebraic property, that may have though a geometric interpretation on a corresponding graph. We shall see in the next sections what is the geometric version of the power of a matrix. Property 3 For every pair of indices i and there exists a natural number m such that ( A m) i is strictly positive. Remark. Going back to a previous example, the matrix ( ) is not irreducible as it is easy to see that the (2,1) element of every power of the matrix is zero. Conversely, the matrix ( ) is irreducible as ( ) = 1 0 ( ) and then if i we have ( A m) > 0 with m = 1 and if i = we have ( A m) > 0 with m = 2. i i We remark that property 3 does not mean of course that there is a power of A that is a positive matrix. Property 4 The n n non-negative matrix A can be associated to a direct graph G, with n nodes, having an edge from node i to node if and only if a i > 0. The matrix is irreducible if and only if the graph G is strongly connected. Example. Let s consider again the example, already seen before, in Figure A = with associated graph v 1 v 4 Figure 6: Adacency and connectedness The graph is strongly connected and A is irreducible. We may use Property 1: it is not difficult to see that for each coordinate subspace we have in R 4, none of them is invariant. If we consider instead the example in Figure 7 it easy to see that the graph is not strongly connected as for example there is no way to go from v 2 to v 1. The adacency matrix is not irreducible, as for example x 1 0 A 0 0 = A = with associated graph v 1 v 4 Figure 7: Adacency and connectedness 12

14 Sometimes it is easier to decide if a matrix is irreducible by giving a look at the corresponding graph, but if the graph is very large and complicated it is reasonable to think it may be easier to study the algebraic aspects of the matrix. Before we go into the details of the thorem in the Frobenius form, the concept of period of a matrix is important. Let A be a non-negative square matrix. Definition 11 Given an index i, the period of index i is the greatest common divisor of all natural numbers m such that ( A m) ii > 0. Example. Let s take the permutation matrix we ve used before A = By observing that A 2 = = A T and A 3 = A A 2 = A A T = I 3 3, we may conclude that for each index i = 1,2,3 the period of index i is 3. When A is irreducible the period of every index is the same and is called the period of the matrix A. In the previous example the matrix is not irreducible, but the indexes are the same however. The matrix A = is irreducible (its graph is strongly connected) and we have A 2 = 0 1 1, A 3 = 1 1 0, A 4 = 1 1 1, A 5 = It means that the period is 1. Definition 12 The matrix A is aperiodic if its period is one. There can be of course an interpretation of the period of a matrix in terms of the properties of the corresponding graph. An interesting characterization exists involving irreducible, aperiodic non-negative matrices. We need another definition first. Definition 13 A matrix A is primitive if it is non-negative and its m-th power is positive for some natural number m. 5 The characterization is the following: Theorem 2 A matrix A is primitive if and only if A is non-negative, irreducible and aperiodic. The last example shows a primitive matrix. Here is the Perron-Frobenius theorem for non-negative irreducible matrices. 6 5 Sometimes primitive matrices are defined as non-negative matrices having only one eigenvalue r = ρ(a) in the spectral circle. It means of course that it is real and simple. In this way our definition can be converted in a theorem: a non-negative matrix A is primitive if and only if there exists m > 0 such that A m is positive. 6 There are actually some other possible statements associated to the Perron-Frobenius theorem that we do not mention. 13

15 Theorem 3 (Perron-Frobenius for non-negative irreducible matrices) Let A be a non-negative irreducible n n matrix with period p and spectral radius ρ(a) = r. Then the following statements hold. (i) r is positive and it is an eigenvalue of the matrix A, called the Perron-Frobenius eigenvalue. (ii) r is simple. Both right and left eigenspaces associated with r are one-dimensional. (iii) A has a left eigenvector v and a right eigenvector w associated with r, whose components are positive for both and the only eigenvectors with all positive components are the ones associated with r. (iv) The matrix A has exactly p (the period) complex eigenvalues with module r. Each of them is a simple root of the characteristic polynomial and is the product of r with a p-th complex root of the unity. ( ) k A (v) lim = vw T, where the right and left eigenvectors are normalized, so that w T v = 1. Moreover, the k + r matrix vw T is the proection onto the eigenspace V r, the so called Perron proection. (vi) The Perron-Frobenius eigenvalue r satisfies the inequalities min i a i r max i a i. To conclude this section we mention another result regarding primitive matrices. Theorem 4 If A in non-negative and irreducible, with r = ρ(a), A is primitive if and only if In this case lim k + lim k + ( ) k A exists. r ( ) k A = pqt r q T p, where p and q T are respectively a right and a left Perron-Frobenius vectors. 3 Some elements from Markov chains theory For the use of the Markov chain model in the PageRank Algorithm a very simple approach to Markov chains is sufficient. So we will skip the general definition that involves the probabilistic concept of a general random process. We may ust think of n possible states of a system, for which the times t = 1,2,3,... are relevant. At each time the system can be in one (and only one) of its states. To give ust a bit of formalization, we may think of the random variables X t, t = 1,2,3,..., where each one may take its values in the set of states E = {1,2,...,n}. We say that at time t the system is in its k-th state if the random variable X t assumes the value k. Of course this will happen with probability Prob(X t = k). Our system is a Markov chain if the following property holds: for each t = 1,2,3,..., for each pair of states i, and for every sequence of states ν 0,ν 1,...,ν t 1 E Prob ( X t+1 = X0 = ν 0, X 1 = ν 1,..., X t 1 = ν t 1, X t = i ) = Prob ( X t+1 = Xt = i ). The meaning is: the probability the system goes from state i to state at time t is independent on the states the system was in before time t. 14

16 For example, taking t = 1, Prob ( X 2 = X0 = ν 0, X 1 = i ) = Prob ( X 2 = Xt = i ), that is the transition at time t = 1 does not depend on the initial state ν 0, but depends only on the state i at that time. The conditional probability Prob ( X t+1 = Xt = i ) = p i (t) is called the probability of transition at time t from state i to state. Of course this probability may or may not depend on t. The Markov chain is said to be homogeneous or time stationary if the probability of transition does not depend on t, namely p i (t) = p i for each t. It is clear that we can associate both a matrix (independent on t) and a graph to a stationary Markov chain. The matrix P = (p i ) is called the transition matrix of the chain and it has the following properties: 1. p i 0 for each i,; 2. n =1 p i = 1 for each i. A matrix with properties 1 and 2 is called a stochastic matrix. Remark. Property 2 may be written in matrix form as Pu = u, where u = (1,...,1) T. This means that any stochastic matrix has the eigenvalue 1 and u is a corresponding eigenvector. It can be proved that stochastic matrices have spectral radius 1. It is quite natural to associate the transition matrix P of a Markov chain to a direct graph. In fact P is a sort of adacency matrix, with an extra information: it tells not ust which transitions are possible but also what is the probability of any single transition. Given the square transition matrix P = (p i ) of order n, the associated graph has n nodes {v 1,...,v n } and has the edge (v i,v ) if and only if p i > 0. It is what is called a weighted graph, a graph in which each edge is a positively weighted edge, weights being probabilities in this case. Of course a state is reachable from a state i if and only if there exists a direct path from v i to v in the graph. Definition 14 A Markov chain is irreducible if its transition matrix P is irreducible. Remark. Among the various characterizations of irreducible matrices we have seen before it means that the graph is strongly connected, namely for any two nodes v i and v there is a direct path that goes from v i to v. Definition 15 A Markov chain is aperiodic if it is irreducible and its transition matrix P is primitive. Let s call probability vector any non-negative row vector p T = (p 1,...,p n ) such that n i=1 p i = 1. Definition 16 A probability vector p is a stationary probability vector for a Markov chain with transition matrix P if p T P = p T. 15

17 Remark. It comes out here the importance for P to be irreducible. In fact from the Perron-Frobenius theorem in that case the existence and uniqueness of a stationary probability vector is guaranteed. This is a crucial aspect in the PageRank algorithm. It is clear that the probability vectors are the right theoretical ways to describe the system as time goes on. We have ust to formalize the way the probability vectors change in time. Suppose we have an initial probability vector p T (0) = (p 1 (0),...,p n (0)). This vector gives the probabilities the system is in the state 1,...,n at t = 0. In the same way p T (t) = (p 1 (t),...,p n (t)) will be the probability vector at time t: it gives the probabilities the system is in the state 1,...,n at time t. Now, given the probabilities at time t, let s compute the probability that the system is in the state at time t+1. ( (Xt+1 p (t+i) = Prob(X t+1 = ) = Prob = ) ( X t = 1... X t = n )) ( (Xt+1 = Prob = X t = 1 )... ( X t+1 = X t = n )) = = = n Prob ( X t+1 = X t = i ) i=1 n Prob ( X t+1 = Xt = i ) Prob ( X t = i ) i=1 n p i p i (t). i=1 It means that the -th component of the row vector p T (t + 1) is given by the inner product of the p T (t) vector itself by the -th column of the transition matrix P. In matrix-vector form p T (t+1) = p T (t)p. It means that for any time t = 1,2,... we may write p T (t) = p T (0)P t. As we shall see in the next subsection this is the characteristic main step of the power method. 4 The PageRank algorithm The main step of Google s PageRank algorithm is the iteration of the method we describe in the following subsection. 4.1 The power method For the purpose of finding the dominant eigenvalue of a matrix together with a corresponding eigenvector a great variety of methods have been designed. This method is a fundamental step in the Google s PageRank algorithm. Let s consider a square n n matrix A having n linearly independent eigenvectors associated to the eigenvalues λ 1,λ 2,...,λ n and suppose λ 1 > λ 2... λ n. 16

18 This means we are assuming that the dominant eigenvalue λ 1 has algebraic multiplicity one. Let s call v 1,v 2,...,v n the corresponding eigenvectors. This means Av i = λ i v i, i = 1,2,...,n. The power method, starting from any arbitrarily chosen vector x (0), builds up a sequence of vectors {x (k) } that converges to the eigenvector associated to the dominant eigenvalue. Suppose x (0) is an arbitrary vector in R n. We may write x (0) as a linear combination of v 1,v 2,...,v n that, because the hypothesis of linear independence, is a basis of R n. n x (0) = α i v i and suppose α i=1 Starting from x (0) we may build up the sequence x (1) = Ax (0), x (2) = Ax (1),..., x (k) = Ax (k 1),... The following result holds: Theorem 5 For the sequence {x (k) } k N we have that x (k+1) lim = λ k x (k) 1 and lim k x (k) x (k) = cv 1. (1) where is an index for which x (k) 0, for every value of k. Proof 1 From the definition of the sequence {x (k) } we have By collecting λ k 1 we get and then n x (k) = Ax (k 1) = A 2 Ax (k 2) =... = A k x (0) = A k α i v i = Then for the indices for which x (k) As λi λ 1 < 1 for 2 i n, we have x (k) = λ k 1 x (k+1) = λ k+1 1 ( ( α 1 v 1 + α 1 v 1 + n i=2 n i=2 i=1 ( ) ) k λi α i v i λ 1 0 and v 1 0 we may write x (k+1) x (k) = λ 1 α 1 v 1 + n i=2 α i α 1 v 1 + n i=2 α i x (k+1) lim = λ k x (k) 1. n α i A k v i = i=1 ( ) ) k+1 λi α i v i. λ 1 ( λ i λ 1 ) k+1v i n α i λ k iv i. i=1 ( λ i λ 1 ) kv i. (2) 7 The condition means that we don t have to start from a point in the subspace spanned by the eigenvectors v 2,v 3,...,v n. We need x (0) to have a component in the subspace spanned by v 1. 17

20 Of course this setting assumes that if P is linked to P i, the transition from P to P i is ust one of the P possible transitions, each one with the same probability. The reason why the dominant eigenvalue and its corresponding eigenvector are important in the PageRank algorithm is going to appear now. Suppose we collect the ranks of all the web pages in a vector r and suppose we want to compute it in an iterative way, starting from a previous evaluation of r itself. At the beginning the most reasonable setting, or maybe the only possible one, is to assume that all the pages have the same rank and, as probabilities are implicitly involved, r = r (0) is the uniform distribution vector r (0) = ( 1 n, 1 n,..., 1 ) T. 8 n Let s rewrite equation (3) in terms of the n pages P 1,P 2,...,P n of the web, and in particular for the i-th page P i, after the first transition: r (1) i (P) = (0) r P P i P = P P i 1 P r(0) = n =1 a i r (0) i = 1,2,...,n. In matrix/vector form the previous equation may be written as r (1) = Ar (0). Clearly an iterative sequence is then defined, namely r (k+1) = Ar (k) k = 0,1,... and this is the general iteration of the power method. We have seen that the limit of the sequence is an eigenvector associated to the dominant eigenvalue. The ranks of the web pages are the components of this limit vector, conveniently normalized. Remark. A couple of immediate remarks. Firstly it appears evident the extreme heaviness of the computations. The number of the web pages exceeds a couple of billions and this is the length of the rank vector r and the order of the matrix A. A more theoretical aspect is that the existence and the properties of the dominant eigenvalue depend on some properties of the matrix A, and for the moment we do not know if the transition matrix has those properties. We shall go into these details in a later work. References L. R. Foulds, Graph Theory Applications, Sringer-Verlag, 1992 G. H. Golub, C.F. Van Loan, Matrix Computations, The Johns Hopkins University Press, 1996 J. R. Norris, Markov chains, Cambridge University Press, 1997 PageRank, at and related pages 8 Transposition is required in the next vector/matrix notations. 19

Calculating Web Page Authority Using the PageRank Algorithm

Jacob Miles Prystowsky and Levi Gill Math 45, Fall 2005 1 Introduction 1.1 Abstract In this document, we examine how the Google Internet search engine uses the PageRank algorithm to assign quantitatively

A Note on Google s PageRank

A Note on Google s PageRank According to Google, google-search on a given topic results in a listing of most relevant web pages related to the topic. Google ranks the importance of webpages according to

Markov Chains and Stochastic Sampling

Part I Markov Chains and Stochastic Sampling 1 Markov Chains and Random Walks on Graphs 1.1 Structure of Finite Markov Chains We shall only consider Markov chains with a finite, but usually very large,

Eigenvalues and Eigenvectors

LECTURE 3 Eigenvalues and Eigenvectors Definition 3.. Let A be an n n matrix. The eigenvalue-eigenvector problem for A is the problem of finding numbers λ and vectors v R 3 such that Av = λv. If λ, v are

Chapter 4 & 5: Vector Spaces & Linear Transformations

Chapter 4 & 5: Vector Spaces & Linear Transformations Philip Gressman University of Pennsylvania Philip Gressman Math 240 002 2014C: Chapters 4 & 5 1 / 40 Objective The purpose of Chapter 4 is to think

4. Linear transformations as a vector space 17

4 Linear transformations as a vector space 17 d) 1 2 0 0 1 2 0 0 1 0 0 0 1 2 3 4 32 Let a linear transformation in R 2 be the reflection in the line = x 2 Find its matrix 33 For each linear transformation

CS168: The Modern Algorithmic Toolbox Lecture #8: How PCA Works

CS68: The Modern Algorithmic Toolbox Lecture #8: How PCA Works Tim Roughgarden & Gregory Valiant April 20, 206 Introduction Last lecture introduced the idea of principal components analysis (PCA). The

(a) II and III (b) I (c) I and III (d) I and II and III (e) None are true.

1 Which of the following statements is always true? I The null space of an m n matrix is a subspace of R m II If the set B = {v 1,, v n } spans a vector space V and dimv = n, then B is a basis for V III

1 9/5 Matrices, vectors, and their applications

1 9/5 Matrices, vectors, and their applications Algebra: study of objects and operations on them. Linear algebra: object: matrices and vectors. operations: addition, multiplication etc. Algorithms/Geometric

Markov Chains, Stochastic Processes, and Matrix Decompositions

Markov Chains, Stochastic Processes, and Matrix Decompositions 5 May 2014 Outline 1 Markov Chains Outline 1 Markov Chains 2 Introduction Perron-Frobenius Matrix Decompositions and Markov Chains Spectral

Graph Models The PageRank Algorithm

Graph Models The PageRank Algorithm Anna-Karin Tornberg Mathematical Models, Analysis and Simulation Fall semester, 2013 The PageRank Algorithm I Invented by Larry Page and Sergey Brin around 1998 and

Linear vector spaces and subspaces.

Math 2051 W2008 Margo Kondratieva Week 1 Linear vector spaces and subspaces. Section 1.1 The notion of a linear vector space. For the purpose of these notes we regard (m 1)-matrices as m-dimensional vectors,

Chapters 5 & 6: Theory Review: Solutions Math 308 F Spring 2015

Chapters 5 & 6: Theory Review: Solutions Math 308 F Spring 205. If A is a 3 3 triangular matrix, explain why det(a) is equal to the product of entries on the diagonal. If A is a lower triangular or diagonal

The Google Markov Chain: convergence speed and eigenvalues

U.U.D.M. Project Report 2012:14 The Google Markov Chain: convergence speed and eigenvalues Fredrik Backåker Examensarbete i matematik, 15 hp Handledare och examinator: Jakob Björnberg Juni 2012 Department

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

Affine iterations on nonnegative vectors

Affine iterations on nonnegative vectors V. Blondel L. Ninove P. Van Dooren CESAME Université catholique de Louvain Av. G. Lemaître 4 B-348 Louvain-la-Neuve Belgium Introduction In this paper we consider

Math 304 Handout: Linear algebra, graphs, and networks.

Math 30 Handout: Linear algebra, graphs, and networks. December, 006. GRAPHS AND ADJACENCY MATRICES. Definition. A graph is a collection of vertices connected by edges. A directed graph is a graph all

Calculating determinants for larger matrices

Day 26 Calculating determinants for larger matrices We now proceed to define det A for n n matrices A As before, we are looking for a function of A that satisfies the product formula det(ab) = det A det

Definition (T -invariant subspace) Example. Example

Eigenvalues, Eigenvectors, Similarity, and Diagonalization We now turn our attention to linear transformations of the form T : V V. To better understand the effect of T on the vector space V, we begin

Ir O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v )

Section 3.2 Theorem 3.6. Let A be an m n matrix of rank r. Then r m, r n, and, by means of a finite number of elementary row and column operations, A can be transformed into the matrix ( ) Ir O D = 1 O

How works. or How linear algebra powers the search engine. M. Ram Murty, FRSC Queen s Research Chair Queen s University

How works or How linear algebra powers the search engine M. Ram Murty, FRSC Queen s Research Chair Queen s University From: gomath.com/geometry/ellipse.php Metric mishap causes loss of Mars orbiter

Eigenvalues and Eigenvectors

Eigenvalues and Eigenvectors Philippe B. Laval KSU Fall 2015 Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 1 / 14 Introduction We define eigenvalues and eigenvectors. We discuss how to

Google PageRank. Francesco Ricci Faculty of Computer Science Free University of Bozen-Bolzano

Google PageRank Francesco Ricci Faculty of Computer Science Free University of Bozen-Bolzano fricci@unibz.it 1 Content p Linear Algebra p Matrices p Eigenvalues and eigenvectors p Markov chains p Google

Linear Algebra- Final Exam Review

Linear Algebra- Final Exam Review. Let A be invertible. Show that, if v, v, v 3 are linearly independent vectors, so are Av, Av, Av 3. NOTE: It should be clear from your answer that you know the definition.

Link Analysis Leonid E. Zhukov School of Data Analysis and Artificial Intelligence Department of Computer Science National Research University Higher School of Economics Structural Analysis and Visualization

Lecture 10. Lecturer: Aleksander Mądry Scribes: Mani Bastani Parizi and Christos Kalaitzis

CS-621 Theory Gems October 18, 2012 Lecture 10 Lecturer: Aleksander Mądry Scribes: Mani Bastani Parizi and Christos Kalaitzis 1 Introduction In this lecture, we will see how one can use random walks to

Linear Algebra in Actuarial Science: Slides to the lecture

Linear Algebra in Actuarial Science: Slides to the lecture Fall Semester 2010/2011 Linear Algebra is a Tool-Box Linear Equation Systems Discretization of differential equations: solving linear equations

Google Page Rank Project Linear Algebra Summer 2012

Google Page Rank Project Linear Algebra Summer 2012 How does an internet search engine, like Google, work? In this project you will discover how the Page Rank algorithm works to give the most relevant

Chapter Two Elements of Linear Algebra

Chapter Two Elements of Linear Algebra Previously, in chapter one, we have considered single first order differential equations involving a single unknown function. In the next chapter we will begin to

THEORY OF SEARCH ENGINES. CONTENTS 1. INTRODUCTION 1 2. RANKING OF PAGES 2 3. TWO EXAMPLES 4 4. CONCLUSION 5 References 5

THEORY OF SEARCH ENGINES K. K. NAMBIAR ABSTRACT. Four different stochastic matrices, useful for ranking the pages of the Web are defined. The theory is illustrated with examples. Keywords Search engines,

18.06SC Final Exam Solutions

18.06SC Final Exam Solutions 1 (4+7=11 pts.) Suppose A is 3 by 4, and Ax = 0 has exactly 2 special solutions: 1 2 x 1 = 1 and x 2 = 1 1 0 0 1 (a) Remembering that A is 3 by 4, find its row reduced echelon

Summer Session Practice Final Exam

Math 2F Summer Session 25 Practice Final Exam Time Limit: Hours Name (Print): Teaching Assistant This exam contains pages (including this cover page) and 9 problems. Check to see if any pages are missing.

Singular Value Decomposition

Chapter 6 Singular Value Decomposition In Chapter 5, we derived a number of algorithms for computing the eigenvalues and eigenvectors of matrices A R n n. Having developed this machinery, we complete our

MATH 20F: LINEAR ALGEBRA LECTURE B00 (T. KEMP)

MATH 20F: LINEAR ALGEBRA LECTURE B00 (T KEMP) Definition 01 If T (x) = Ax is a linear transformation from R n to R m then Nul (T ) = {x R n : T (x) = 0} = Nul (A) Ran (T ) = {Ax R m : x R n } = {b R m

Generalized eigenvector - Wikipedia, the free encyclopedia

1 of 30 18/03/2013 20:00 Generalized eigenvector From Wikipedia, the free encyclopedia In linear algebra, for a matrix A, there may not always exist a full set of linearly independent eigenvectors that

Linear Algebra. Carleton DeTar February 27, 2017

Linear Algebra Carleton DeTar detar@physics.utah.edu February 27, 2017 This document provides some background for various course topics in linear algebra: solving linear systems, determinants, and finding

THE MINIMAL POLYNOMIAL AND SOME APPLICATIONS

THE MINIMAL POLYNOMIAL AND SOME APPLICATIONS KEITH CONRAD. Introduction The easiest matrices to compute with are the diagonal ones. The sum and product of diagonal matrices can be computed componentwise

Linear Algebra Massoud Malek

CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product

BASIC ALGORITHMS IN LINEAR ALGEBRA. Matrices and Applications of Gaussian Elimination. A 2 x. A T m x. A 1 x A T 1. A m x

BASIC ALGORITHMS IN LINEAR ALGEBRA STEVEN DALE CUTKOSKY Matrices and Applications of Gaussian Elimination Systems of Equations Suppose that A is an n n matrix with coefficents in a field F, and x = (x,,

eigenvalues, markov matrices, and the power method

eigenvalues, markov matrices, and the power method Slides by Olson. Some taken loosely from Jeff Jauregui, Some from Semeraro L. Olson Department of Computer Science University of Illinois at Urbana-Champaign

0.2 Vector spaces. J.A.Beachy 1

J.A.Beachy 1 0.2 Vector spaces I m going to begin this section at a rather basic level, giving the definitions of a field and of a vector space in much that same detail as you would have met them in a

CS246: Mining Massive Datasets Jure Leskovec, Stanford University

CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu 2/7/2012 Jure Leskovec, Stanford C246: Mining Massive Datasets 2 Web pages are not equally important www.joe-schmoe.com

1. General Vector Spaces

1.1. Vector space axioms. 1. General Vector Spaces Definition 1.1. Let V be a nonempty set of objects on which the operations of addition and scalar multiplication are defined. By addition we mean a rule

Singular Value Decomposition

Singular Value Decomposition (Com S 477/577 Notes Yan-Bin Jia Sep, 7 Introduction Now comes a highlight of linear algebra. Any real m n matrix can be factored as A = UΣV T where U is an m m orthogonal

22m:033 Notes: 7.1 Diagonalization of Symmetric Matrices

m:33 Notes: 7. Diagonalization of Symmetric Matrices Dennis Roseman University of Iowa Iowa City, IA http://www.math.uiowa.edu/ roseman May 3, Symmetric matrices Definition. A symmetric matrix is a matrix

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

MATH 310, REVIEW SHEET 2

MATH 310, REVIEW SHEET 2 These notes are a very short summary of the key topics in the book (and follow the book pretty closely). You should be familiar with everything on here, but it s not comprehensive,

No books, no notes, no calculators. You must show work, unless the question is a true/false, yes/no, or fill-in-the-blank question.

Math 304 Final Exam (May 8) Spring 206 No books, no notes, no calculators. You must show work, unless the question is a true/false, yes/no, or fill-in-the-blank question. Name: Section: Question Points

Math 110 Linear Algebra Midterm 2 Review October 28, 2017

Math 11 Linear Algebra Midterm Review October 8, 17 Material Material covered on the midterm includes: All lectures from Thursday, Sept. 1st to Tuesday, Oct. 4th Homeworks 9 to 17 Quizzes 5 to 9 Sections

Elementary Linear Algebra Review for Exam 2 Exam is Monday, November 16th.

Elementary Linear Algebra Review for Exam Exam is Monday, November 6th. The exam will cover sections:.4,..4, 5. 5., 7., the class notes on Markov Models. You must be able to do each of the following. Section.4

Eigenvalues, Eigenvectors, and Diagonalization

Week12 Eigenvalues, Eigenvectors, and Diagonalization 12.1 Opening Remarks 12.1.1 Predicting the Weather, Again Let us revisit the example from Week 4, in which we had a simple model for predicting the

Applied Mathematics 205. Unit II: Numerical Linear Algebra. Lecturer: Dr. David Knezevic

Applied Mathematics 205 Unit II: Numerical Linear Algebra Lecturer: Dr. David Knezevic Unit II: Numerical Linear Algebra Chapter II.3: QR Factorization, SVD 2 / 66 QR Factorization 3 / 66 QR Factorization

Topics in linear algebra

Chapter 6 Topics in linear algebra 6.1 Change of basis I want to remind you of one of the basic ideas in linear algebra: change of basis. Let F be a field, V and W be finite dimensional vector spaces over

Spectral radius, symmetric and positive matrices

Spectral radius, symmetric and positive matrices Zdeněk Dvořák April 28, 2016 1 Spectral radius Definition 1. The spectral radius of a square matrix A is ρ(a) = max{ λ : λ is an eigenvalue of A}. For an

STA141C: Big Data & High Performance Statistical Computing

STA141C: Big Data & High Performance Statistical Computing Lecture 6: Numerical Linear Algebra: Applications in Machine Learning Cho-Jui Hsieh UC Davis April 27, 2017 Principal Component Analysis Principal

Eigenvalues, Eigenvectors, and Diagonalization

Math 240 TA: Shuyi Weng Winter 207 February 23, 207 Eigenvalues, Eigenvectors, and Diagonalization The concepts of eigenvalues, eigenvectors, and diagonalization are best studied with examples. We will

ALGEBRA QUALIFYING EXAM PROBLEMS LINEAR ALGEBRA

ALGEBRA QUALIFYING EXAM PROBLEMS LINEAR ALGEBRA Kent State University Department of Mathematical Sciences Compiled and Maintained by Donald L. White Version: August 29, 2017 CONTENTS LINEAR ALGEBRA AND

A proof of the Jordan normal form theorem

A proof of the Jordan normal form theorem Jordan normal form theorem states that any matrix is similar to a blockdiagonal matrix with Jordan blocks on the diagonal. To prove it, we first reformulate it

MA 265 FINAL EXAM Fall 2012

MA 265 FINAL EXAM Fall 22 NAME: INSTRUCTOR S NAME:. There are a total of 25 problems. You should show work on the exam sheet, and pencil in the correct answer on the scantron. 2. No books, notes, or calculators

Extra Problems for Math 2050 Linear Algebra I

Extra Problems for Math 5 Linear Algebra I Find the vector AB and illustrate with a picture if A = (,) and B = (,4) Find B, given A = (,4) and [ AB = A = (,4) and [ AB = 8 If possible, express x = 7 as

Properties of Matrices and Operations on Matrices

Properties of Matrices and Operations on Matrices A common data structure for statistical analysis is a rectangular array or matris. Rows represent individual observational units, or just observations,

Lecture 9: SVD, Low Rank Approximation

CSE 521: Design and Analysis of Algorithms I Spring 2016 Lecture 9: SVD, Low Rank Approimation Lecturer: Shayan Oveis Gharan April 25th Scribe: Koosha Khalvati Disclaimer: hese notes have not been subjected

w T 1 w T 2. w T n 0 if i j 1 if i = j

Lyapunov Operator Let A F n n be given, and define a linear operator L A : C n n C n n as L A (X) := A X + XA Suppose A is diagonalizable (what follows can be generalized even if this is not possible -

Real symmetric matrices/1. 1 Eigenvalues and eigenvectors

Real symmetric matrices 1 Eigenvalues and eigenvectors We use the convention that vectors are row vectors and matrices act on the right. Let A be a square matrix with entries in a field F; suppose that

Numerical Methods I: Eigenvalues and eigenvectors

1/25 Numerical Methods I: Eigenvalues and eigenvectors Georg Stadler Courant Institute, NYU stadler@cims.nyu.edu November 2, 2017 Overview 2/25 Conditioning Eigenvalues and eigenvectors How hard are they

AN ELEMENTARY PROOF OF THE SPECTRAL RADIUS FORMULA FOR MATRICES

AN ELEMENTARY PROOF OF THE SPECTRAL RADIUS FORMULA FOR MATRICES JOEL A. TROPP Abstract. We present an elementary proof that the spectral radius of a matrix A may be obtained using the formula ρ(a) lim

1 Searching the World Wide Web

Hubs and Authorities in a Hyperlinked Environment 1 Searching the World Wide Web Because diverse users each modify the link structure of the WWW within a relatively small scope by creating web-pages on

MATH 304 Linear Algebra Lecture 20: The Gram-Schmidt process (continued). Eigenvalues and eigenvectors.

MATH 304 Linear Algebra Lecture 20: The Gram-Schmidt process (continued). Eigenvalues and eigenvectors. Orthogonal sets Let V be a vector space with an inner product. Definition. Nonzero vectors v 1,v

Review and problem list for Applied Math I

Review and problem list for Applied Math I (This is a first version of a serious review sheet; it may contain errors and it certainly omits a number of topic which were covered in the course. Let me know

(a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? Solution: dim N(A) 1, since rank(a) 3. Ax =

. (5 points) (a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? dim N(A), since rank(a) 3. (b) If we also know that Ax = has no solution, what do we know about the rank of A? C(A)

Matrices related to linear transformations

Math 4326 Fall 207 Matrices related to linear transformations We have encountered several ways in which matrices relate to linear transformations. In this note, I summarize the important facts and formulas

Jordan normal form notes (version date: 11/21/07)

Jordan normal form notes (version date: /2/7) If A has an eigenbasis {u,, u n }, ie a basis made up of eigenvectors, so that Au j = λ j u j, then A is diagonal with respect to that basis To see this, let

ft-uiowa-math2550 Assignment OptionalFinalExamReviewMultChoiceMEDIUMlengthForm due 12/31/2014 at 10:36pm CST

me me ft-uiowa-math255 Assignment OptionalFinalExamReviewMultChoiceMEDIUMlengthForm due 2/3/2 at :3pm CST. ( pt) Library/TCNJ/TCNJ LinearSystems/problem3.pg Give a geometric description of the following

Linear Algebra Primer

Linear Algebra Primer David Doria daviddoria@gmail.com Wednesday 3 rd December, 2008 Contents Why is it called Linear Algebra? 4 2 What is a Matrix? 4 2. Input and Output.....................................

Midterm 2 Solutions, MATH 54, Linear Algebra and Differential Equations, Fall 2014

Name (Last, First): Student ID: Circle your section: 2 Shin 8am 7 Evans 22 Lim pm 35 Etcheverry 22 Cho 8am 75 Evans 23 Tanzer 2pm 35 Evans 23 Shin 9am 5 Latimer 24 Moody 2pm 8 Evans 24 Cho 9am 254 Sutardja

Problems in Linear Algebra and Representation Theory

Problems in Linear Algebra and Representation Theory (Most of these were provided by Victor Ginzburg) The problems appearing below have varying level of difficulty. They are not listed in any specific

Math 443/543 Graph Theory Notes 5: Graphs as matrices, spectral graph theory, and PageRank

Math 443/543 Graph Theory Notes 5: Graphs as matrices, spectral graph theory, and PageRank David Glickenstein November 3, 4 Representing graphs as matrices It will sometimes be useful to represent graphs

Majorizations for the Eigenvectors of Graph-Adjacency Matrices: A Tool for Complex Network Design

Majorizations for the Eigenvectors of Graph-Adjacency Matrices: A Tool for Complex Network Design Rahul Dhal Electrical Engineering and Computer Science Washington State University Pullman, WA rdhal@eecs.wsu.edu

EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 2

EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 2 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory April 5, 2012 Andre Tkacenko

An Introduction to Entropy and Subshifts of. Finite Type

An Introduction to Entropy and Subshifts of Finite Type Abby Pekoske Department of Mathematics Oregon State University pekoskea@math.oregonstate.edu August 4, 2015 Abstract This work gives an overview

In particular, if A is a square matrix and λ is one of its eigenvalues, then we can find a non-zero column vector X with

Appendix: Matrix Estimates and the Perron-Frobenius Theorem. This Appendix will first present some well known estimates. For any m n matrix A = [a ij ] over the real or complex numbers, it will be convenient

Linear Algebra and Eigenproblems

Appendix A A Linear Algebra and Eigenproblems A working knowledge of linear algebra is key to understanding many of the issues raised in this work. In particular, many of the discussions of the details

LECTURE VII: THE JORDAN CANONICAL FORM MAT FALL 2006 PRINCETON UNIVERSITY. [See also Appendix B in the book]

LECTURE VII: THE JORDAN CANONICAL FORM MAT 204 - FALL 2006 PRINCETON UNIVERSITY ALFONSO SORRENTINO [See also Appendix B in the book] 1 Introduction In Lecture IV we have introduced the concept of eigenvalue

NONCOMMUTATIVE POLYNOMIAL EQUATIONS. Edward S. Letzter. Introduction

NONCOMMUTATIVE POLYNOMIAL EQUATIONS Edward S Letzter Introduction My aim in these notes is twofold: First, to briefly review some linear algebra Second, to provide you with some new tools and techniques

Stat 159/259: Linear Algebra Notes

Stat 159/259: Linear Algebra Notes Jarrod Millman November 16, 2015 Abstract These notes assume you ve taken a semester of undergraduate linear algebra. In particular, I assume you are familiar with the

B553 Lecture 5: Matrix Algebra Review

B553 Lecture 5: Matrix Algebra Review Kris Hauser January 19, 2012 We have seen in prior lectures how vectors represent points in R n and gradients of functions. Matrices represent linear transformations

0.1 Eigenvalues and Eigenvectors

0.. EIGENVALUES AND EIGENVECTORS MATH 22AL Computer LAB for Linear Algebra Eigenvalues and Eigenvectors Dr. Daddel Please save your MATLAB Session (diary)as LAB9.text and submit. 0. Eigenvalues and Eigenvectors

Example Linear Algebra Competency Test

Example Linear Algebra Competency Test The 4 questions below are a combination of True or False, multiple choice, fill in the blank, and computations involving matrices and vectors. In the latter case,

MATH 304 Linear Algebra Lecture 34: Review for Test 2.

MATH 304 Linear Algebra Lecture 34: Review for Test 2. Topics for Test 2 Linear transformations (Leon 4.1 4.3) Matrix transformations Matrix of a linear mapping Similar matrices Orthogonality (Leon 5.1

Eigenvalues and Eigenvectors

CHAPTER Eigenvalues and Eigenvectors CHAPTER CONTENTS. Eigenvalues and Eigenvectors 9. Diagonalization. Complex Vector Spaces.4 Differential Equations 6. Dynamical Systems and Markov Chains INTRODUCTION

Lecture 11: Eigenvalues and Eigenvectors

Lecture : Eigenvalues and Eigenvectors De nition.. Let A be a square matrix (or linear transformation). A number λ is called an eigenvalue of A if there exists a non-zero vector u such that A u λ u. ()

The QR Decomposition

The QR Decomposition We have seen one major decomposition of a matrix which is A = LU (and its variants) or more generally PA = LU for a permutation matrix P. This was valid for a square matrix and aided

Tangent spaces, normals and extrema

Chapter 3 Tangent spaces, normals and extrema If S is a surface in 3-space, with a point a S where S looks smooth, i.e., without any fold or cusp or self-crossing, we can intuitively define the tangent

Recall the convention that, for us, all vectors are column vectors.

Some linear algebra Recall the convention that, for us, all vectors are column vectors. 1. Symmetric matrices Let A be a real matrix. Recall that a complex number λ is an eigenvalue of A if there exists

EXERCISE SET 5.1. = (kx + kx + k, ky + ky + k ) = (kx + kx + 1, ky + ky + 1) = ((k + )x + 1, (k + )y + 1)

EXERCISE SET 5. 6. The pair (, 2) is in the set but the pair ( )(, 2) = (, 2) is not because the first component is negative; hence Axiom 6 fails. Axiom 5 also fails. 8. Axioms, 2, 3, 6, 9, and are easily

6 EIGENVALUES AND EIGENVECTORS

6 EIGENVALUES AND EIGENVECTORS INTRODUCTION TO EIGENVALUES 61 Linear equations Ax = b come from steady state problems Eigenvalues have their greatest importance in dynamic problems The solution of du/dt

Math 4153 Exam 3 Review. The syllabus for Exam 3 is Chapter 6 (pages ), Chapter 7 through page 137, and Chapter 8 through page 182 in Axler.

Math 453 Exam 3 Review The syllabus for Exam 3 is Chapter 6 (pages -2), Chapter 7 through page 37, and Chapter 8 through page 82 in Axler.. You should be sure to know precise definition of the terms we

Gershgorin s Circle Theorem for Estimating the Eigenvalues of a Matrix with Known Error Bounds

Gershgorin s Circle Theorem for Estimating the Eigenvalues of a Matrix with Known Error Bounds Author: David Marquis Advisors: Professor Hans De Moor Dr. Kathryn Porter Reader: Dr. Michael Nathanson May