Exercise 1.3 Work out the probabilities that it will be cloudy/rainy the day after tomorrow.

Transcription

1 54 Chapter Introduction 9 s Exercise 3 Work out the probabilities that it will be cloudy/rainy the day after tomorrow Exercise 5 Follow the instructions for this problem given on the class wiki For the example described in this section, Recreate the above table by programming it up with Matlab or Octave, starting with the assumption that today is cloudy 2 Create two similar tables starting with the assumption that today is sunny andrainy, respectively 3 Compare how x 7 differs depending on today s weather 4 What do you notice if you compute x k starting with today being sunny/cloudy/rainy and you let k get large? 5 What does x represent? Exercise 6 Given Table, create the following table, which predicts the weather the day after tomorrow given the weather today: Day after Tomorrow sunny cloudy rainy This then tells us the entries in Q in 7 Today sunny cloudy rainy Exercise 8 Let x, y R n Show that vector addition commutes: x + y y + x Exercise 9 Let x, y R n Show that x T y y T x

2 9 s 55 Exercise Start building your library by implement the functions in Figure 6 See directions on the class wiki page Exercise 8 Use mathematical induction to prove that n i i2 n n2n /6 : Proof by induction: Base case: n For this case, we must show that i i2 22 /6 i i2 Definition of summation arithmetic 2 /6 This proves the base case Inductive step: Inductive Hypothesis IH: Assume that the result is true for n k where k : k i 2 k k2k /6 i We will show that the result is then also true for n k + : k+ i i 2 k + k2k + /6 Assume that k Then

3 56 Chapter Introduction k+ i i 2 arithmetic k i i2 split off last term k i i2 + k 2 Inductive Hypothesis k k2k /6+k 2 algebra 2k 3 3k 2 + k+6k 2 /6 algebra 2k 3 +3k 2 + k/6 algebra kk + 2k + /6 arithmetic k + k + 2k + /6 This proves the inductive step By the Principle of Mathematical Induction the result holds for all n Exercise 24 For each of the following, determine whether it is a linear transformation or not: χ χ F F χ χ 2 F χ χ First check if F F transformation So it COULD be a linear

4 9 s 57 Next, check if F αx αf x Let x an arbitrary scalar Then χ F α F αχ α α χ be an arbitrary vector and α be αχ α Next, check if F x + y F x+fy Let x arbitrary vectors Then χ F + χ ψ ψ ψ 2 + Thus it is a linear transformation χ χ 2 F First check if F F transformation F ψ ψ 2 F αf χ χ + ψ + ψ + ψ 2 χ χ and y + F ψ ψ ψ 2 χ + ψ + ψ 2 ψ ψ 2 be an So it COULD be a linear Now, looking at it, I suspect this is not a linear transformation because of the So, I will try to construct an example where F αx αf x or F x + y F x+fy Let x and α 2 Then 2 4 F αx F 2 F Also 2F x 2F 2 2 Thus, for this choice of α and x we find that F αx αf x Thus it is not a linear transformation

5 58 Chapter Introduction Exercise 26 Let x, e i R n Show that e T i x x T e i χ i the ith element of x Recall that e i where the occurs in the ith entry and x χ χ i χ i χ i+ χ n Then e T i x χ + + χ i + χ i + χ i+ + + χ n χ i Exercise 34 Show that the transformation in Example 2 is not a linear transformation by computing a possible matrix that represents it, and then showing that it does not represent it χ χ + ψ Let F If F were a linear transformation, then there would be ψ χ + α β a corresponding matrix, A, such that γ δ χ α β χ F ψ γ δ ψ This matrix would be computed by computing its columns α + F γ + 2 and α β so that γ δ β F δ But 2 2 χ ψ χ + ψ 2χ + ψ + + χ + ψ χ + Thus F cannot be a linear transformation There is no matrix that has the same action as F

6 9 s 59 Exercise 35 Show that the transformation in Example 23 is not a linear transformation by computing a possible matrix that represents it, and then showing that it does not represent it χ χψ Let F If F were a linear transformation, then there would be a ψ χ α β corresponding matrix, A, such that γ δ χ α β χ F ψ γ δ ψ This matrix would be computed by computing its columns α F γ and α β so that γ δ β F δ But χ ψ χ χψ χ Thus F cannot be a linear transformation There is no matrix that has the same action as F Exercise 36 For each of the transformations in Exercise 24 compute a possible matrix that represents it and use it to show whether the transformation is linear Let F χ χ a corresponding matrix, A F χ If F were a linear transformation, then there would be α α α 2 α α α 2 α 2 α 2 α 22, such that α α α 2 α α α 2 α 2 α 2 α 22 χ

7 6 Chapter Introduction This matrix would be computed by computing its columns α α F, α 2 and so that α α α 2 α α α 2 α 2 α 2 α 22 α α α 2 α 2 α 2 α 22 F F χ Checking now χ,, F χ Thus there is a matrix that corresponds to F which is therefore a linear transformation χ χ 2 Let F If F were a linear transformation, then there would be α α a corresponding matrix, A, such that α α χ α α F χ α α This matrix would be computed by computing its columns α F and α α α α α so that α α χ F Checking now Thus it is not a linear transformation χ F χ

8 9 s 6 Exercise 39 Let D 2 3 represent? In particular, answer the following questions: L : R? R?? Give? and?? What linear transformation, L, does this matrix A linear transformation can be describe by how it transforms the unit basis vectors: Le Le Le 2 χ L L : R 3 R 3 A linear transformation can be describe by how it transforms the unit basis vectors: 2 Le Le 3 Le 2 L χ Lχ e + e + e 2 χ Le + Le + Le 2 χ χ 3 Thus, the elements of the input vector are scaled by the corresponding elements of the diagonal of the matrix

9 62 Chapter Introduction Exercise 42 Give examples for each of the triangular matrices in Definition 4 lower triangular strictly lower triangular unit lower triangular upper triangular strictly upper triangular unit upper triangular if α i,j for all i<j if α i,j for all i j if α i,j for all i<jand α i,j if i j if α i,j for all i>j if α i,j for all i j if α i,j for all i>jand α i,j if i j Exercise 43 Show that a matrix that is both lower and upper triangular is in fact a diagonal matrix If a matrix is both upper and lower triangular then α ij if i j Thus the matrix is diagonal Exercise 44 Add the functions trilu A and triuu A to your SLAP library These functions return the lower and upper triangular part of A, respectively, with the diagonal set to ones Thus, > A [ ]; > trilu A ans Hint: use the tril and eye functions You will also want to use the size function to extract the dimensions of A, to pass in to eye

10 9 s 63 See the base directory of the SLAP library Exercise 49 Show that a triangular matrix that is also symmetric is in fact a diagonal matrix Let us focus on a lower triangular matrix Then α ij if i>j But if the matrix is also symmetric then α ij if i<j Thus α ij if i j Exercise 56 Prove Theorem 22 We need to show that L C αx αl C x: L C αx L A αx+l B αx αl A x+αl B x αl C x L C x + y L C x+l C y: L C x + y L A x + y+l B x + y L A x+l A y+l B x+l B y L A x+l B x+l A y+l B y L C x+l C y Exercise 58 Note: I have changed this question from before Give a motivation for matrix addition by considering a linear transformation L C x L A x+l B x Given A, B, C R m n let L A, L B and L C be the corresponding linear transformations Define L C x L A x +L B x Let a j, b j,and c j be the jth columns of A, B, and C, respectively Then c j L C e j L A e J +L B e j a j + b j Thus the elements of the each of the columns of C equal the addition of the corresponding elements of columns of A and B Exercise 67 Modify the algorithms in Figure 9 to compute y : Lx, where L is a lower triangular matrix The answer is in Figure 23 It computes y : Lx + y, since the given operation can then be obtained by starting with y the vector of zeroes

11 64 Chapter Introduction y : Ltrmvmult unb varba, x, y ATL A TR Partition A, A BL A BR yt x, y y B where A TL is, x T, y T are while ma TL <ma do Repartition ATL A TR A BL A BR x A a A 2 a T α a T 2 A 2 a 2 A 22, yt y B where α,, and ψ are scalars y ψ, y : Ltrmvmult unb var2ba, x, y ATL A TR Partition A, A BL A BR yt x, y y B where A TL is, x T, y T are while ma TL <ma do Repartition ATL A TR A BL A BR x A a A 2 a T α a T 2 A 2 a 2 A 22, yt y B where α,, and ψ are scalars y ψ, ψ : a T x + α +a T 2 }{{} + ψ {}}{ y : a + y ψ : α + ψ : a 2 + Continue with ATL A TR A BL endwhile A BR x A a A 2 a T α a T 2 A 2 a 2 A 22, yt y B y ψ, Continue with ATL A TR A BL endwhile A BR x A a A 2 a T α a T 2 A 2 a 2 A 22, yt y B y ψ, Figure 23: to Exercise 67 Compare and contrast to Figure 9 Executing either of these algorithms with A L will compute y : Lx + y where L is lower triangular Notice that the algorithm would become even clearer if {A, a, α} were replaced by {L, l, λ} Exercise 68 Modify the algorithms in Figure 2 so that x is overwritten with x : Ux, without using the vector y The answer is in Figure 24 It computes x : Ux Some explaination: Let us first consider y : Ux + y If we partition the matrix and vectors we get y ψ : U u U 2 υ u T 2 U 22 x + y ψ U x + u + U 2 + y υ + u T 2 + ψ U 2 +

12 9 s 65 Notice that the algorithm in Figure 2left has the property that when the current iteration starts, everything in red currently exists in y: y U x + u + U 2 + y ψ : υ + u T 2 + ψ U 2 + The update ψ : υ + u T 2 + ψ then makes it so that one more element of y has been updated Now, turn to the computation x : Ux: x U x + u + U 2 : υ + u T 2 U 2 Notice that to update one more element of x we much compute : υ + u T 2 which justifies the algorithm in Figure 24left Now, let s turn to the algorithm in Figure 2right That algorithm has the property that when the current iteration starts, everything in red currently exists in y: y ψ : U x + u + U 2 +y υ + u T 2 + ψ U 2 + The updates y : u + y ψ : υ + ψ then make it so that the vector y contains everything in blue: y U x + u + U 2 +y ψ : υ + u T 2 +ψ U 2 + Now, turn to the computation x : Ux: x U x + u + U 2 : υ + u T 2 U 2 Notice that the computations x : u + x : υ

13 66 Chapter Introduction x : Trmv un unb varbu, x UTL U TR Partition U, U BR x where U TL is, x T is while mu TL <mu do Repartition UTL U TR U BR x where υ, are scalars : u T x + υ + u T 2 U u U 2 υ u T 2 U 22, x : Trmv un unb var2u, x UTL U TR Partition U, U BR x where U TL is, x T is while mu TL <mu do Repartition UTL U TR U BR x where υ, are scalars x : u + x : υ U u U 2 υ u T 2 U 22, Continue with UTL U TR U BR endwhile x U u U 2 υ u T 2 A 22, Continue with UTL U TR U BR endwhile x U u U 2 υ u T 2 A 22, Figure 24: to Exercise 67 make is so that everthing in blue is in x: x U x + u + U 2 : υ + u T 2 U 2 which justifies the algorithm in Figure 24right Exercise 69 Reason why the algorithm you developed for Exercise 67 cannot be trivially changed so that x : Lx without requiring y What is the solution? Let us focus on the algorithm on the left in Figure 23 Consider the update ψ : a T x + α +a T 2 }{{} + ψ

14 9 s 67 and reason what would happen if we blindly changed this to : a T x + α Then x on the right of the : refers to the original contents of subvector x, but those have been overwritten by previous iterations The solution is to make this change, but to then move through the matrix backwards This is not an issue for the case where we are working with an upper triangular matrix, because there the computation accidently moves in just the right direction You may want to clarify this to yourself by working through a small 3 3 example Exercise 7 Develop algorithms for computing x : U T x and x : L T x, where U and L are respectively upper triangular and lower triangular, without explicitly transposing matrices U and L Let us focus on computing x : U T x Compare U u U 2 υ u T 2 U 22 T U T u T υ U2 T u 2 U22 T to L l T λ L 2 l 2 L 22 Notice that computing x : U T x is the same as computing x : Lx except that you use U T for L, u T for l T, etc Thus, you need to make the obvious changes to the algorithms you developed for x : Lx Ditto for x : L T x except that you modify the algorithms you developed for x : Ux Exercise 7 Compute the cost, in flops, of the algorithm in Figure 2right Let us analyze the algorithm in Figure 2right The cost is in the updates and y : u + y ψ : υ + ψ Now, during the first iteration, u and y are of length, so that that iteration requires 2 flops for the second step only During the ith iteration staring with i, u and y are

15 68 Chapter Introduction of length i so that the cost of that iteration is 2i flops for the first step an axpy operation and 2 flops for the second Thus, if U is an n n matrix, then the total cost is given by n i n [2 + 2i] 2n +2 i nn i 2n +2 2n + n 2 n n 2 + n n 2 2 flops Thus the cost of the two different algorithms is the same! This is not surprising: to get from one algorithm to the other, all you are doing is reordering the same operations Exercise 74 Modify the algorithms in Figure 9 to compute y : Ax + y, where A is symmetric and stored in the lower triangular part of matrix Notice that if A is symmetric, then A a A 2 a T α a T 2 A 2 a 2 A 22 which means that so that A T a A T 2 a T α a T 2 A T 2 a 2 A T 22 T A a A 2 a T α a T 2 A 2 a 2 A 22 A a A 2 a T α a T 2 A 2 a 2 A 22 A T A a a A T 2 A 2 a T a T α a T 2 a T 2 A T 2 A 2 a 2 a 2 A T 22 A 22 Thus, if only the lower triangular part is stored, we can take advantage of the equalities in Figure 9, replacing in the left algorithm A T A a T a T α A T 2 A 2 a 2 a 2 A T 22 A 22 ψ : a T + α + a T 2 + ψ by ψ : a T + α + a T 2 + ψ and in the right algorithm y : a + y ψ : α + ψ : a 2 + by y : a + y ψ : α + ψ : a 2 +

16 9 s 69 Exercise 77 Follow the directions on the wiki to implement the above routines for computing the general matrix-vector multiply Exercise 78 Follow the directions on the wiki to implement the above routines for computing the triangular matrix-vector multiply Exercise 79 Follow the directions on the wiki to implement the above routines for computing the symmetric matrix-vector multiply Exercise 8 When you try to pick from the algorithms in Figure 2, you will notice that neither algorithm has the property that almost all computations involve data that is stored consecutively in memory There are actually two more algorithms for computing this operation The observation is that y : Ax + y, where A is symmetric and stored in the upper trianglar part of the array, can be computed via y : Ux + y followed by y : Û T x + y, where U and Û are the upper triangular and strictly upper triangular parts of A Now, for the first there are two algorithms given in the text, and the other one is an exercise This gives us 2 2 combinations By merging the two loops that you get one for each of the two operations you can find all four algorithms for the symmetric matrix-vector multiplication From these, you can pick the algorithm that strides through memory in the most favorable way