Lab 4: Working with arrays and eigenvalues

Transcription

1 Lab 4: Working with arrays and eigenvalues March 23, 2017 The purpose of this lab is to get familiar with manipulating data arrays in numpy. Browser use: This lab introduces the use of a number of Python packages that are particularly useful for working with images such as *.jpg pictures taken with a cell phone. These packages include skimage and PIL (Python Imaging Library). We strongly recommend installing Canopy Express on your computer: if so, then these packages can be readily installed using the Package Manager to install scikits-image. Unfortunately many of these packages are not well documented and hence the only way to learn about using them is to use these packages and consult the Internet: Chances are if you are having a problem, then someone else has posted a question on the Internet about it and hopefully someone else has answered it! In addition, Appendices C and D will be useful as references. Housekeeping: In your home directory make two directories pyprogs and pyimages. It is convenient to put the programs we write in this lab into the directory pyporgs since we will use them in future labs. The directory pyimages will be used to store the programs that we will develop to manipulate images such as *.jpg pictures. 1 Background You can activate python in command mode, by typing the word python at the command prompt, then hit ENTER. In Canopy Express the command mode window is opened when you type Editor (it s the lower of the two windows that open). It is useful to do this to follow the discussion. Now type 1

2 import numpy as np then ENTER, so that you have access to the numpy library of commands. Here are two very important concepts in scientific computing that are often under-emphasized: 1. Not all numbers are the same: There are three basic number types: integers, floating point number and complex numbers. An integer is a number such as 1, 2, 3, etc. Note that integers do not have a decimal point. A floating point number is a number that has a decimal point, i.e 1.0, , and so on. A complex number has the form floating point number ± j floating point number. In Python we can specify the data type by using the command dtype. Thus it we want our number to be short integers we could specify dtype=np.int8 if a floating point number dtype=np.float32 and so on. For imaging we most often use dtype=unit8. This notation stands for 8-byte, unsigned integers. Why are images limited to 8-bytes? There are two reasons: 1). Most computer monitors can only display 8 bytes, and 2) The human eye is only able to render about 8 bytes of visual information. For a useful list of the possible dtypes see If we keep in mind that computers store objects as 0 s and 1 s it is not hard to imagine that we can specify dtypes that are not numbers. For example, dtype=str means a string of letters like a word and dtype=bool refers to either True or False. We can even use the command dtype=object to refer to something that is none of the above. 2. An important data structure in numpy is the array. For an extensive discussion of arrays in Python see Appendix C and using 2

3 arrays in numpy see Appendix D. Here we briefly touch on some of the important points. An array is a table of objects (usually numbers for scientists) having n rows and m columns. For scientists it is convenient to think of an array as a table of data. For example let us consider that the position of the hand during throwing a baseball has been recorded by motion capture cameras. The output of the camera system is typically a table of data in which the first column is time, the second column is the position of the hand in the x=direction at time t, the third column is the position in the y-direction at time t and the fourth column is the z position of the hand at time t. Thus data array has n rows, where n is the nukber of time samples, and 4 columns. The shape of this data array is (n, 4). Suppose for simplicity that we have the array stored in the file test.dat The shape of this array is 3 rows and 2 columns. We can load this data set into our program using the command X = np.loadtxt( test.dat ) The command np.shape(x) tells us the shape of X and returns (3,2). We can also use the command reshape() to change the shape of X. Thus Y=np.reshape(X,(6,1))) returns

4 3. Creating arrays: In the laboratory setting, the most common way to create an array in a Python program is to read in data using the np.loadtxt() command. In this case data file itself determines the way that the numbers are represented, the number of rows and the number of columns. The question is how to create an array of a given shape inside a computer program. In other words for a situation in which we do not read in a data file as discussed above. The answer to this question is important because we often use a computer simulation of a model to predict what the behavior of the experimental observations. Since at some point we will need to compare prediction to observation we need to be able to make an array that contains the predictions of the model. There are three ways to create an array and these are very similar in philosophy The philosophy is the same: we create an array of the desired shape filled with either 0 s, 1 s or random numbers and then replace these numbers by the number generated by our computer simulations. (a) np.zeros(): In this case we create an arrays of 0 s of a required shape. For example, the command X=np.zeros((2,2),dtype=np.float32) creates the array (b) The command np.ones() works in a similar way, except that instead of 1 s we get 0 s. (c) The command np.random.rand() creates an array of a given shape of random floating point numbers, e.g. np.random.rand(2,2) produces

5 It should be noted that for np.zeros() and np.ones() we can specify the dtype, but for np.random.rand() the dtype is always a floating point number. 4. Extracting data from arrays: Each element in an array occupies position (i, j) where i specifies the row and j specifies the column. Thus if we wanted to know what is the value located in the second row, third position of array X we would type X[2,3]. Note that an array position is enclosed by square brackets (in contrast in MATLAB we would use round brackets, i.e. X(2,3)). Keep in mind that the first element in the first row is X[0, 0]. In many languages including Python, the colon : is very special. By itself it denotes all of the corresponding row or column. Thus if we wanted to extract the first column from array X we would type x0=x[:,0] and if we wanted to first row we would type x0=x[0,:] A colon : separating two numbers a : b means rows (columns) a to b. Thus [1:5] means from 1 to 5 Here are some other useful array manipulations: [1:] means 1 to end [len(a):1] means from length a to end [::n] means each n-th item in entire sequence Of course we can also combine these notations, for example, if X was an array with shape (10, 10), then X[1:5,7:10] would mean the first five rows and the last four columns. 5

6 5. Saving computer generated arrays as data files: Numpy lets you write arrays into files in two ways. In order to use these tools well, it is critical to understand the difference between a text and a binary file containing numerical data. In a text file, the number π (referred to as pi in numpy) could be written as , for example: a string of digits that a human can read, with in this case 15 decimal digits. In contrast, that same number written to a binary file would be encoded as 8 characters (bytes) that are not readable by a human but which contain the exact same data that the variable pi had in the computer s memory. The tradeoffs between the two modes are thus: Text mode: occupies more space, precision can be lost (if not all digits are written to disk), but is readable and editable by hand with a text editor. Can only be used for one- and two-dimensional arrays. Binary mode: compact and exact representation of the data in memory, can t be read or edited by hand. Arrays of any size and dimensionality can be saved and read without loss of information. Here we discuss how to read and write arrays in text mode. Appendix D discusses how to read and write arrays in binary mode. The np.savetxt function saves an array to a text file, with options to control the precision, separators and even adding a header. Suppose we have an array X containing 10 numbers from 0 to 9 with shape (2,5) and we want to write it to a file called test.out. The command is np.savetxt( test.out, X, fmt= %1.3f, header="my dataset") and X is # My dataset

7 This is because we used the format option, fmt= %1.2f to specify a floating point number (hence the f ), where the.2 indicates two decimal points. If we wanted the numbers to be in exponential format (like in MATLAB), we would choose fmt= %.2e to obtain # My dataset 0.00e e e e e e e e e e+00 We can read the data using np.loadtxt, namely X=np.loadtxt( test.out ) If we write the command print X, we obtain Why does the file look this way? There are two reasons. First, the default data type for np.loadtxt() is float. Second, comment lines started with # are ignored. If we want to change these defaults, we can look up the options for np.loadtxt() on the Internet. 2 Exercise 1: Stability using XPPAUT In lecture we emphasized three concepts for determining the qualitative behavior of the differential equations we use to model biological phenomena: 1) fixed points, 2) the characteristic equation, and 3) local stability. Briefly, the fixed points are the values of the variables for which all derivatives are equal to zero and local stability refers to the response of the dynamical system to a small perturbation away from the fixed point. Mathematically, the stability of the fixed point is obtained by first linearizing the differential equations using Taylor s theorem and then using the usual Ansatz that x(t) e λt, to obtain a characteristic equation in terms of the eigenvalue, λ. The roots of the characteristic equation corresponds to the eigenvalues that we use to evaluate stability: the fixed point is stable if Re(λ) < 0 and unstable if Re(λ) > 0. 7

8 Figure 1: We obtained the classification of the fixed point and the eigenvalues by clicking on the tab Sing pts after we had changed the axes to make a phase plane portrait.. Note that that the classification of the eigenvalues and the location of the fixed point are shown in the window that opens, but the actual values of the eigenvalues are printed out in the Terminal window. Performing all of these steps becomes increasingly difficult as the number of variables included in the model increases. Since realistic biological models often contain many variables, it quickly happens that we are not able to evaluate stability analytically. Thus recourse to computer programming tools becomes necessary. One possibility is to use a program such as XPPAUT. XPPAUT automatically determines the eigenvalues from the phase plane representation. To see how this is done consider the equation for a damped harmonic oscillator we studied in Lab 2. Construct the phase plane representation and then click on Sing pts (Figure 1). A new window opens which shows the fixed point (bottom), gives the stability (upper right) and classifies the eigenvalues. The XPPAUT convention for classifying the eigenvalues (shown in panel 8

9 at upper left in figure) is c : number of complex eigenvalues with negative real part c + : number of complex eigenvalues with positive real part r : number of real negative eigenvalues r : number of real positive eigenvalues Im : number of pure imaginary eigenvalues The values of the eigenvalues are printed in the window that you used to start XPPAUT. Compare these values to the ones that you determined mathematically. Exercise to do: Use XPPAUT to run one of the *.ode programs you wrote last lab. Once you have a solution, click on Sing pts. Are the fixed points and the eigenvalues values the same as you calculated using paper and pencil? 3 Exercise 2: Stability using Python When we integrate a differential equation using numpy it is necessary to rewrite it as a system of first-order differential equations, for example, dx dt dy dt = ax + by, (1) = cx + dy, where the a, b, c, d are constants. In class we determined the eigenvalues by first reconstructing the differential equation from (1) d 2 x (a + d)dx + (ad bc)x = 0, dt2 dt and then obtaining the λ s (eignevalues) from the characteristic equation λ 2 (a + d)λ + (ad bc) = 0. (2) We adopted this procedure because a course in linear algebra is not a prerequisite for this course. However, this procedure rapidly becomes tedious as 9

10 the number, n, of first-order differential equations increases, i.e. dx 1 = c 11 x 1 + c 12 x 2 + c 13 x c 1,n, dt dx 2 = c 21 x 1 + c 22 x 2 + c 23 x c 2,n, (3) dt. =, dx n = c n1 x 1 + c n2 x 2 + c n3 x c n,n. dt In fact we know that when n > 4 it is unlikely we will be able to determine the λ s analytically. Thus we must resort to numerical procedures. The package np.linalg makes it easy to determine the eigenvalues for a system of linear differential equations of arbitrary size. In order to understand this we need to learn a little bit of linear algebra. First we re-write (3) as dx 1 /dt c 11 c c 1n x 1 dx 2 /dt... dx n /dt = c 21 c c 2n c n1 c n2... c nn x 2... x n. (4) Equation (4) contains two n 1 arrays, one in which the elements are dx i /dt and one in which the elements are x i, and one n n array containing the coefficients. The n 1 arrays corresponds to vectors and the n n array containing n rows and n columns corresponds to a matrix. Using this notation we can re-write (1) as ( ) ( ) ( ) dx/dt a b x =. (5) dy/dt c d y When we multiply a vector by a matrix we multiply a row of the matrix by the vector element by element. Thus we multiple the first element of the first row of the matrix by the first element of the vector, then the second element of the first row of the matrix by the second element of the vector, and so on. In this way way you can see that (5) is an alternate way of writing (1). We can also write (4) as dx dt y = Ax, (6) 10

11 where the bold-face type is used to indicate matrices (capital letter) and vectors (small letters). Equation (6) describes a linear transformation that maps a given array x into a new array y. This statement will become much clearer when we do the Glass-Moiré dot exercise (below). The important point for our discussion is that the λ s (eigenvalues) can be determined from the matrix. Important point: Although it is useful to think of vectors and matrices as kinds of arrays (this in fact is what the programming language MATLAB does), in Python vectors and arrays are not simply kinds of arrays, especially when it comes to multiplication. In other words in Python an array is not the same as a matrix. The package numpy itself contains the package numpy.linalg which makes available a number of useful functions for working with vectors and matrices. Our interest in this package stems from the fact that there is a function, eig(), which outputs the eigenvalues (also the eigenvectors). The important point is that eig() is not restricted to 2 2-matrices, but can be used to determine the eigenvalues for n n matrices where n 2. In order to use these functions it is necessary to remember two things: 1) the subscripts of the coefficients, c ij indicate, respectively, the row number (i) and column (j) number, and 2) a matrix is constructed row by row as np.matrix([[x11, x12,.., xn],.., [xn1, xn2,.., xnn]]) Our discussion up to this point concerns only linear differential equations. For nonlinear differential equations it will be necessary to linearize the equations about a fixed point before we can use numpy.linalg.eig() to determine the eigenvalues. It is posssible to write a computer program that does all of these steps (in fact XPPAUT incorporates this). However, this is an advanced topic. Note that in our typical applications, the number of rows equals the number of columns. The output of numpy.linalg.eig() is two nd.array s: vals gives the λ s and vects gives the eigenvectors. Here our interest here is on the eigenvalues. Example: Write a computer program to determine the eigenvalues, λ s, for dx dt dy dt = y = 3x 2y 11

12 Figure 2: A Moiré-Glass Pattern that resembles the flow patterns near a saddle point. The two colors distinguish the position of the random rots before and after the 2D-plane was rotated by θ. Answer: A Python program that determines the eigenvalues for a system of linear differential equations is import numpy as np import numpy.linalg as linalg X = np.matrix([[0.,1.],[3.,-2.]]) vals, vects = linalg.eig(x) print vals The result is λ = 1, 3. Question: Does this result agree with what you obtained using paper and pencil? 4 Exercise 3: Moiré-Glass dot patterns In the two-dimensional xy-plane, the values of x and y specify the coordinates of a point. A common problem concerns the determination of the coordinates 12

13 of a point when the xy-plane is rotated by an angle θ. From introductory physics we know that the answer to this problem is solved using a rotation matrix, R, [3] where ( ) x = R y new ( ) x y old ( ) cos(θ) sin(θ) R = sin(θ) cos(θ), (7) is the rotation matrix. It has been observed that when a random dot pattern is scaled, rotated, and superimposed over the original data, interesting visual patterns, known as Moiré-Glass Patterns, emerge [1, 2]. Moiré-Glass dot patterns are actively studied in psychophysics to understand how the nervous system perceives visual (sensory) stimuli. A 2-D random dot pattern can be generated using np.rand() which generates random numbers drawn a uniform probability distribution. Thus we can apply transformations to the random dot field, X 1, using a scale, S, and rotation, R, namely X 2 = SRX 1 where ( ) sx 0 S = 0 s y is the scaling matrix. If the scale and rotation factors are small, the transformation is analogous to a single step in the numerical solution of a second order ordinary differential equation, namely, the discrete equation ( ) x(t + 1) y(t + 1 = SR ( ) x(t) y(t), (8) where we have written out X 1 and X 2 in terms of their (x, y) components. The plot of both X 1 and X 2 will reveal the structure of the vector field flow around the fixed point. The fixed point corresponds to the invariant point under the transformation. The eigenvalues of the transformation matrix M = SR determine the nature of the fixed point. We have not discussed how to multiple two matrices and determine the eigenvalues. However, we can generate the Moiré Glass patterns by keeping in mind that the matrix S determines the real part of the eigenvalues and the matrix R determines the imaginary part. If there is no rotation, then the eigenvalues are real and equal to s x and s y. If s x, s y are 13

14 both negative the fixed point is astable node and if they have opposite signs the fixed point is a saddle point. If there is a rotation, then the eigenvalues are complex. If the real parts of both eigenvalues are both less that 1 1. and the eigenvalues are complex, the fixed point is a stable spiral 2. If s x = s y = 0 and there is a rotational componet, we have a center. The program moire_glass,py generates a Moiré-Glass dot pattern that resembles the vector flows in the vicinity of a saddle point (Figure 2). Find values of s x, s y and θ so that the dot pattern resembles a node, a spiral and a center. Two things to keep in mind are 1) the values of these parameters should be quite small since the patterns arise in the local vicinity of the fixed point, and 2) in a discrete dynamical the important point for stability is whether or not λ is less than or greater than 1. In our application this condition becomes whether s x, s y are less than 1. Questions to be answered: 1. Using this information, determine values of s x, s y, θ in the program to obtain the vector field flow for a center and a spiral point. Save these figures as a *.png file. 2. For a given type of Moiré-Glass pattern what happens if you increase the values of s x, s y, θ. For example, does the patterns become more or less evident? 3. What happens if you change the colors of the dots? What happens if you replace the dots with other symbols? 5 Exercise 4: Working with digital images The images produced by a digital camera are most often in *.jpg format. The data in a *.jpg file are presented in the form of an array and thus we can apply the same tricks that we applied to the Moiré dot patterns to digital photographs. Indeed, an infamous felon recently altered his own picture in this way to brag about a crime he had performed, and displayed the picture 1 In a discrete dynamical the condition for stability is that λ < 1 2 In the older literature a spiral point was referred to as a focus. 14

15 Figure 3: A *.jpg picture on the Mona Lisa (left) that is swirled (middle) and then reverse swirled (right). on YouTube. This felon did not realize that an image which has been changed by coordinate transformations, can be changed back to the original. Thus the crook had unwittingly published his confession to the crime for all to see! Skimage (a.k.a. scikit-image) is a collection of algorithms useful for image processing and computer vision. Here we use some of the tools in Skimage to create a swirled image similar to that that the felon used. Image swirling is a non-linear image deformation that creates a whirlpool effect (Figure 3). This image was produced using the program swirl_prog.py. This Python program represents a jump in programming skills from the programs we have considered up to now and thus we do not expect you to be able to digest all of the issues at this point in time. Here we discuss the basic concepts involved in swirling an image. When applying a geometric transformation on an image, we typically make use of a reverse mapping, i.e., for each pixel in the output image, we compute its corresponding position in the input. The reason is that, if we were to do it the other way around (map each input pixel to its new output position), some pixels in the output may be left empty. On the other hand, each output coordinate has exactly one corresponding location in (or 15

16 outside) the input image, and even if that position is non-integer, we may use interpolation to compute the corresponding image value. To perform a reverse mapping, namely a geometric warp, in skimage, you need to provide the reverse mapping to the skimage.transform.warp function. For example, consider the case where we would like to shift an image 50 pixels to the left. The reverse mapping for such a shift would be def shift_left(xy): xy[:, 0] += 50 return xy The corresponding call to warp is: from skimage.transform import warp warp(image, shift_left) In order to understand the swirl transformation, consider the coordinate (x, y) in the output image. The reverse mapping for the swirl transformation first computes, relative to a center, (x 0, y 0 ) using polar coordinates θ = arctan(y/x) ρ = (x x 0 ) 2 + (y y 0 ) 2 and then transforms them according to r = ln(2) radius/5 φ = rotation s = strength θ = φ + s e ρ/r+θ where strength is a parameter for the amount of swirl, radius indicates the swirl extent in pixels, and rotation adds a rotation angle. The transformation of radius into r is to ensure that the transformation decays to 1/1000 th within the specified radius. The program swirl_prog.py, takes a *.jpg picture, mona_lisa.jpg and applies the swirl algorithm. Questions to be answered: 16

17 1. Take a picture with your cell phone and see if you can swirl part of the image using swirl_prog.py. In its present form swirl_prog.py swirls a region centered on the center of the picture. See if you can change this program so that it swirls some other region of the image. Save these as a *.png file. Deliverable: Use Lab4_template.tex to prepare the lab assignment. References [1] I. Amidor (2003). Glass patterns as Moiré effects: new surprising results. Optics Lett. 28: 7-9. [2] L. Glass (1969). Moiré effect from random dots. Nature 223: [3] J. B. Marion (1970). Classical dynamics of particles and systems, 2nd edition. Academic Press, New York,