Coding the Matrix! Divya Padmanabhan. Intelligent Systems Lab, Dept. of CSA, Indian Institute of Science, Bangalore. June 28, 2013

Transcription

1 Coding the Matrix! Divya Padmanabhan Intelligent Systems Lab, Dept. of CSA, Indian Institute of Science, Bangalore June 28, 203 Divya Padmanabhan Coding the Matrix! June 28, 203

2 How does the most popular search engine work? Google it! Search more than 8 billion pages.. Results in less than a second.. How is it possible??? What is Google doing in that second? Divya Padmanabhan Coding the Matrix! June 28, 203 2

3 How does Facebook manage information about you? Friend network matrix U U n U U 2. U n Divya Padmanabhan Coding the Matrix! June 28, 203 3

4 How does Flipkart recommend products for you? How does Flipkart know what you require? How does Flipkart keep track of so many users and find their interests? Divya Padmanabhan Coding the Matrix! June 28, 203 4

5 Some BASICS!!! Divya Padmanabhan Coding the Matrix! June 28, 203 5

6 Preliminaries Vectors eg. How well can you sing? paint? play?.. Professional singer : ( ) ( ). Amateur : 0. Divya Padmanabhan Coding the Matrix! June 28, 203 6

7 Preliminaries Vectors eg. How well can you sing? paint? play?.. Professional singer : ( ) ( ). Amateur : 0. Norm x = x 2 + x x n 2 Dot product x. y = x y cos θ = x y + x 2 y x n y n Divya Padmanabhan Coding the Matrix! June 28, 203 6

8 Orthogonality and Projections Figure: Orthogonal vectors x T y = x y cos θ = 0 x θ y Divya Padmanabhan Coding the Matrix! June 28, 203 7

9 Orthogonality and Projections Figure: Orthogonal vectors x T y = x y cos θ = 0 x θ y Figure: Projection of x on y = x. y y x y Ç x. y y Divya Padmanabhan Coding the Matrix! June 28, 203 7

10 Linear Combination and Independence Linear Combination c x + c 2 y, c R, c 2 R Linear Independence of vectors x and y c x + c 2 y = 0 c = 0, c 2 = 0 Divya Padmanabhan Coding the Matrix! June 28, 203 8

11 Linear Combination and Independence Linear Combination c x + c 2 y, c R, c 2 R Linear Independence of vectors x and y c x + c 2 y = 0 c = 0, c 2 = 0 Can t express x in terms of y. Divya Padmanabhan Coding the Matrix! June 28, 203 8

12 Linear Combination and Independence Linear Combination c x + c 2 y, c R, c 2 R Linear Independence of vectors x and y c x + c 2 y = 0 c = 0, c 2 = 0 Can t express x in terms of y. [ ] [ ] 2 Eg.,, c 0 3 = 0, c 2 = 0 Linearly [ ] [ ] dependent : 2,, c 2 4 = 2, c 2 = Divya Padmanabhan Coding the Matrix! June 28, 203 8

13 Matrix Multiplication Divya Padmanabhan Coding the Matrix! June 28, 203 9

14 Matrix Multiplication A {[ }}{{}}{ [ ] {[ }}]{ = ] B C [ ] [ ] [ ] = Divya Padmanabhan Coding the Matrix! June 28, 203 9

15 Row and column vectors A = [ 2 ] Divya Padmanabhan Coding the Matrix! June 28, 203 0

16 Row and column vectors [ ] 2 0 A = Row vectors = r =, r2 = 0 4 Divya Padmanabhan Coding the Matrix! June 28, 203 0

17 Row and column vectors [ ] 2 0 A = Row vectors = r =, r2 = 0 4 { [ ] [ ] 2 Column vectors = c =, c2 =, c3 = 3 [ ]} 0 4 Divya Padmanabhan Coding the Matrix! June 28, 203 0

18 Fundamental spaces Row space and column space of A Row Space: Linear combination of r, r2. Column space : Linear combination of c, c2, c3 Null space: Vectors orthogonal to row space of A. Row Space (A) = Col space (A T ) Divya Padmanabhan Coding the Matrix! June 28, 203

19 Representatives of a space Basis and dimensions of a space Basis : Smallest no. of linearly independent vectors whose linear combination gives the entire space. [ ] 2 = 2 [ ] + 0 [ ] 0 Dimension : No. of vectors in the basis. Divya Padmanabhan Coding the Matrix! June 28, 203 2

20 Redundancy elimination Rank and nullity of matrix A Rank(A)= no. of linearly independent column vectors of A. Nullity(A) = dimension of nullspace of A. Rank measures redundancy in the matrix. Divya Padmanabhan Coding the Matrix! June 28, 203 3

21 Redundancy elimination Rank and nullity of matrix A Rank(A)= no. of linearly independent column vectors of A. Nullity(A) = dimension of nullspace of A. Rank measures redundancy in the matrix. [ ] Divya Padmanabhan Coding the Matrix! June 28, 203 3

22 Redundancy elimination Rank and nullity of matrix A Rank(A)= no. of linearly independent column vectors of A. Nullity(A) = dimension of nullspace of A. Rank measures redundancy in the matrix. [ ] Rank-Nullity Theorem: Rank(A) + Nullity(A) = n Divya Padmanabhan Coding the Matrix! June 28, 203 3

23 Eigen values and eigen vectors ÈÙ Ð ËÔ Ò ¼º x ½ Ê Ö Divya Padmanabhan Coding the Matrix! June 28, 203 4

24 Eigen values and eigen vectors ÈÙ Ð ËÔ Ò ¼º x A 2 2 ½ Ê Ö Divya Padmanabhan Coding the Matrix! June 28, 203 4

25 Eigen values and eigen vectors ÈÙ Ð ËÔ Ò ÈÙ Ð ËÔ Ò ¼º x A 2 2 ½º ¼º x Ax = λx ½ Ê Ö ½ ¾ Ê Ö Divya Padmanabhan Coding the Matrix! June 28, 203 4

26 Eigen values and eigen vectors ÈÙ Ð ËÔ Ò ÈÙ Ð ËÔ Ò ¼º x A 2 2 ½º ¼º x Ax = λx ½ ½ ¾ Ê Ö Ê Ö How do you determine the x and λ given a matrix A? Ax = λx, x 0 ( or x T A T = λx T, x 0 ) Divya Padmanabhan Coding the Matrix! June 28, 203 4

27 Eigen values and eigen vectors ÈÙ Ð ËÔ Ò ÈÙ Ð ËÔ Ò ¼º x A 2 2 ½º ¼º x Ax = λx ½ Ê Ö ½ ¾ Ê Ö How do you determine the x and λ given a matrix A? Ax = λx, x 0 ( or x T A T = λx T, x 0 ) (A λi )x = 0 Divya Padmanabhan Coding the Matrix! June 28, 203 4

28 Eigen values and eigen vectors ÈÙ Ð ËÔ Ò ÈÙ Ð ËÔ Ò ¼º x A 2 2 ½º ¼º x Ax = λx ½ Ê Ö ½ ¾ Ê Ö How do you determine the x and λ given a matrix A? Ax = λx, x 0 ( or x T A T = λx T, x 0 ) (A λi )x = 0 det(a λi ) = 0 Divya Padmanabhan Coding the Matrix! June 28, 203 4

29 Eigen values and eigen vectors ÈÙ Ð ËÔ Ò ÈÙ Ð ËÔ Ò ¼º x A 2 2 ½º ¼º x Ax = λx ½ Ê Ö ½ ¾ Ê Ö How do you determine the x and λ given a matrix A? Ax = λx, x 0 ( or x T A T = λx T, x 0 ) (A λi )x = 0 det(a λi ) = 0 Divya Padmanabhan Coding the Matrix! June 28, 203 4

30 Now for the FUN stuff!!! Divya Padmanabhan Coding the Matrix! June 28, 203 5

31 A Typical Search Activity : Random surfer Divya Padmanabhan Coding the Matrix! June 28, 203 6

32 A Typical Search Activity : Random surfer Jump Divya Padmanabhan Coding the Matrix! June 28, 203 6

33 Importance of pages Divya Padmanabhan Coding the Matrix! June 28, 203 7

34 Importance of pages Divya Padmanabhan Coding the Matrix! June 28, 203 7

35 Page Rank Algorithm Divya Padmanabhan Coding the Matrix! June 28, 203 8

36 Page Rank Algorithm Transition Matrix H A B C D A B C D Divya Padmanabhan Coding the Matrix! June 28, 203 8

37 Page Rank Algorithm Anything particular about H? Transition Matrix H A B C D A B C D Divya Padmanabhan Coding the Matrix! June 28, 203 8

38 Page Rank Algorithm Anything particular about H? H : row stochastic Transition Matrix H A B C D A B C D Divya Padmanabhan Coding the Matrix! June 28, 203 8

39 Page Rank Algorithm Anything particular about H? H : row stochastic Transition Matrix H A B C D A B C D V t (A) : Probability of being at page A at time t V t : row vector. eg. [ ] Divya Padmanabhan Coding the Matrix! June 28, 203 8

40 Page Rank Algorithm Anything particular about H? H : row stochastic Transition Matrix H A B C D A B C D V t (A) : Probability of being at page A at time t V t : row vector. eg. [ ] V t (A) = V t (A)H AA + V t (B)H BA +... Divya Padmanabhan Coding the Matrix! June 28, 203 8

41 Page Rank Algorithm Anything particular about H? H : row stochastic Transition Matrix H A B C D A B C D V t (A) : Probability of being at page A at time t V t : row vector. eg. [ ] V t (A) = V t (A)H AA + V t (B)H BA +... V t = V t H Divya Padmanabhan Coding the Matrix! June 28, 203 8

42 Page Rank Algorithm Transition Matrix H A B C D A B C D H : row stochastic, guaranteed to have eigen value, under some conditions. /4 V 0 = /4 /4, /4 Divya Padmanabhan Coding the Matrix! June 28, 203 9

43 Page Rank Algorithm Transition Matrix H A B C D A B C D H : row stochastic, guaranteed to have eigen value, under some conditions. /4 9/24 V 0 = /4 /4, V = V 0 H = 5/24 5/24, /4 5/24 Divya Padmanabhan Coding the Matrix! June 28, 203 9

44 Page Rank Algorithm Transition Matrix H A B C D A B C D H : row stochastic, guaranteed to have eigen value, under some conditions. /4 9/24 5/48 V 0 = /4 /4, V = V 0 H = 5/24 5/24, V 2 = V H = /48 /48 /4 5/24 /48 Divya Padmanabhan Coding the Matrix! June 28, 203 9

45 Page Rank Algorithm Transition Matrix H A B C D A B C D H : row stochastic, guaranteed to have eigen value, under some conditions.. V n = V n H Divya Padmanabhan Coding the Matrix! June 28,

46 Page Rank Algorithm Transition Matrix H A B C D A B C D H : row stochastic, guaranteed to have eigen value, under some conditions.. V n = V n H Divya Padmanabhan Coding the Matrix! June 28,

47 Page Rank Algorithm Transition Matrix H A B C D A B C D H : row stochastic, guaranteed to have eigen value, under some conditions.. V n = V n H. vh = v Divya Padmanabhan Coding the Matrix! June 28,

48 Page Rank Algorithm Transition Matrix H A B C D A B C D H : row stochastic, guaranteed to have eigen value, under some conditions.. V n = V n H. vh = v v is the eigen vector of H corresponding to eigen value!!! Divya Padmanabhan Coding the Matrix! June 28,

49 Page Rank Algorithm - conditions on H Transition Matrix H A B C D A B C D Divya Padmanabhan Coding the Matrix! June 28, 203 2

50 Page Rank Algorithm - conditions on H Row stochastic Transition Matrix H A B C D A B C D Divya Padmanabhan Coding the Matrix! June 28, 203 2

51 Page Rank Algorithm - conditions on H Row stochastic Transition Matrix H A B C D A B C D Irreducible :Possible to reach a page from any other page. Divya Padmanabhan Coding the Matrix! June 28, 203 2

52 Page Rank Algorithm - conditions on H Row stochastic Transition Matrix H A B C D A B C D Irreducible :Possible to reach a page from any other page. Aperiodic : Not periodic Divya Padmanabhan Coding the Matrix! June 28, 203 2

53 Page Rank Algorithm - Dangling nodes Transition Matrix H A B C D A B C D Divya Padmanabhan Coding the Matrix! June 28,

54 Page Rank Algorithm - Dangling nodes S = H + Y = Transition Matrix H A B C D A B C D A B C D A B C D Divya Padmanabhan Coding the Matrix! June 28,

55 Page Rank Algorithm - Entering a new destination S = A B C D A B C D 4 4 Google matrix! 4 4 /n /n B : /n /n Surfer follows the link structure 60 % of time Otherwise to a random page! G = αs + ( α)b n n Divya Padmanabhan Coding the Matrix! June 28,

56 Page Rank Algorithm Search Engines Get pages containing query words. Use page rank to determine the order of pages to be displayed. Divya Padmanabhan Coding the Matrix! June 28,

57 Page Rank Algorithm Search Engines Get pages containing query words. Use page rank to determine the order of pages to be displayed. Ford 0. Maruti 0.7 Volkswagen 0.05 Hyundai 0.5 Divya Padmanabhan Coding the Matrix! June 28,

58 Recommender Systems Divya Padmanabhan Coding the Matrix! June 28,

59 Recommender Systems Matrix Factorization Original matrix X d n (d users, n movies) X ˆX = M d k H k n Find M, H. Divya Padmanabhan Coding the Matrix! June 28,

60 References Gilbert Strang, Linear Algebra and its Applications, Fourth Edition. 2 Langville and Meyer, Google s PageRank and Beyond. 3 Gene H. Golub, Charles F. Van Loan, Matrix Computations. 4 Howard Anton, Chris Rorres, Elementary Linear Algebra. Divya Padmanabhan Coding the Matrix! June 28,

61 THANK YOU! Divya Padmanabhan Coding the Matrix! June 28,