Stochastic Modelling

Transcription

1 Stochastic Modelling Simulating Random Walks and Markov Chains This lab sheets is available for downloading from as is the spreadsheet mentioned in section 1. The lab sheet was developed using Microsoft Excel for Office 97, but should work on subsequent versions as well. 1 The Pseudo-Random Numbers You can download a spreadsheet from the web page mentioned above, or you can start a new spreadsheet. If you choose to start a new one, you will need a column of pseudo-random numbers for the simulation. Name one of the worksheets Rand and fill the range A1:A200 with a single formula =RAND(). Use Insert Name Define (or equivalent) to call this range Random. [Note: There are various problems with using Excel's pseudo-random number generator: Excel insists on recalculating all its random numbers each time you do anything; the generator is not very sophisticated and does not rate highly in comparison with other generators; it is hard to set seeds in such a way that a sequence of pseudo-random numbers can be reproduced. That is why the spreadsheet supplied on the web page uses a different approach to generate the pseudorandom numbers.] 2 The Simple Random Walk * 1 Start a new worksheet and call it "SRW". The simple random walk has a single parameter, p, so set aside a cell to hold the value, and name the cell p. To start off with, use the value 0.5. Column A will hold the values of the random walk, column B the increments (jumps). So, in cell B5, enter the formula =IF(Random<p, 1, -1), and copy this down into the next 50 cells. Remember that the IF statement takes the form IF(Condition, value if condition true, value if condition false). In A4 enter the starting value (0 will do), then in A5 the formula =A4+B5, which can again be copied down for 50 rows. While this column is still selected, click on Chart Wizard and go through the steps required to obtain a Line chart of the SRW. Change the value of p by a small amount such as ±0.01 and observe what happens to the SRW as shown on the chart. Since we are using the same seed all the time, changes are relatively small. Now change the seed on the Generator sheet (choose any number in the range 0 to 1) and see how the chart changes. 3 Barriers *We shall use columns C and D to construct another SRW, but this time we shall impose some barriers on it. We shall need to store the location and type of each barrier. Type labels UpperLocation, LowerLocation, UpperType and LowerType into cells D2, D3, F2 and F3, and use Insert Name Define to give the adjacent cells (E2, E3, G2, G3) these names. 1 * indicates that this has been done for you on the downloadable spreadsheet. Stochastic Modelling 2003 Session 1, page 1

2 Now decide where you want the barriers to be 0 and 5 would be simple to begin with, so enter these in E3 and E2 respectively. The barrier type will be a numerical value: 1 for a downward reflecting barrier, 0 for an absorbing barrier and 1 for an upward reflecting barrier. Choose whatever types you like and enter them in G2 and G3. The increments, which will be in column D starting from D5, are now generated using a nested IF statement: =IF(C4=UpperLocation,UpperType,IF(C4=LowerLocation,LowerType,IF(Random<p, 1, -1))). Check that you understand how this works, then enter the formula =C4+D5 in cell C5 and copy both formulas down for 50 rows. Select the values in the C column, press Copy, then click on the chart and choose Paste Special from the Edit menu to get the new data series added to the chart along with the original random walk. Try changing the barrier locations to see how much difference they make, both in the case where p = 0.5 and in the case where p is different from 0.5. See if you can predict what you are going to observe before you actually observe it. 4 General discrete distributions for increments Although Excel can be used to simulate from the Binomial and Poisson distributions, we are going to show how to use it to simulate from a more general discrete distribution. *To begin with, we need to set up a table listing the possible values and their probabilities. Start a new worksheet and name it "Discrete". At the top left corner of the new worksheet construct a region like this: Prob CumProb 0 =B1+B2 =C1+C2 =D1+D2 =E1+E2 =F1+F2 Value and give the name "ProbTable" to the range B2:F3. This is called a Lookup Table: we are going to produce random numbers between 0 and 1, look them up in the Cumulative Probability row and return the corresponding quantity from the Value column. The worksheet function to use is HLOOKUP. The syntax of the HLOOKUP function is =HLOOKUP(Lookup value, Lookup table, Row number), where the row number refers to the row of the table to look in, starting from the first row of the table which is counted as row 1. In this case the lookup values are in the first row of ProbTable (column B) and the corresponding values of the variable are in the second row (column C). Therefore a suitable increment for the random walk will be generated by the formula =HLOOKUP(Random, ProbTable, 2). Simulate this random walk for 50 steps and add the resulting trajectory to the diagram. 5 The transition matrix A general discrete-time Markov chain has transition probabilities which are likely to vary depending on the current state. It is not too hard to write a macro in Visual Basic to carry out the transitions, but it is more challenging to try to arrange a simulation mechanism using only the functions which are built into Excel. We shall use look-up tables to accomplish our goals. *Start a new worksheet and call it Markov. Enter a title for the sheet somewhere on row 1. We are going to simulate a no-claims discount scheme with 4 states: No discount, small discount, medium discount and large discount. The transition mechanism is: if there are no claims during a year, the Stochastic Modelling 2003 Session 1, page 2

3 policyholder moves to the next higher level of discount (unless already receiving Large discount); if there is one or more claim during a year the policyholder moves to the next lower discount level (unless receiving no discount). *We need a parameter: type Claim frequency into cell A3, with some value like 0.4 in B3: this represents the average number of claims per year. The Poisson distribution will be used, so the probability of no claims, q, is given by the formula =EXP(-B3): enter this formula in B4, with the label q in A4, and name the cell q (using Insert Name Define). Now enter the transition matrix into a blank area on the worksheet, starting from column E. The diagram below contains labels in column D: they are not necessary, but serve to remind us which state is which. The procedure outlined here only works if the states are numbered with integers starting from 0. Don t forget that every entry must be non-negative and that the row sums must be 1: it s probably best to make the last column contain 1 (sum of previous columns). D E F G H I J 3 No discount 0 =1-q =q 0 =1-SUM(F3:H3) 4 Small disc 1 =1-q 0 =q =1-SUM(F4:H4) 5 Med disc 2 0 =1-q 0 =1-SUM(F5:H5) 6 Large disc =1-q =1-SUM(F6:H6) The lookup function needs a cumulative transition matrix: the table shown below will calculate these in the form required. Use Insert Name Define to name the cumulative matrix (in this example, F8:J11) CumMatrix, and the vector of states (F12:J12) StateVector. D E F G H I J 7 8 No discount 0 0 =F3 =G8+G3 =H8+H3 =I8+I3 9 Small disc 1 0 =F4 =G9+G4 =H9+H4 =I9+I4 10 Med disc 2 0 =F5 =G10+G5 =H10+H5 =I10+I5 11 Large disc 3 0 =F6 =G11+G6 =H11+H6 =I11+I6 12 =E8 =E9 =E10 =E The LOOKUP function The LOOKUP function has syntax LOOKUP(lookup_value, lookup_vector, result_vector). Here the lookup_value is going to be a random number between 0 and 1. We tell Excel which row of the cumulative transition matrix to consult (the row corresponding to the current state, i). When it finds the largest entry in that row which does not exceed the lookup_value it notes how many columns it had to search, then sends back the corresponding entry from the result_vector: this will be the state j to which the chain jumps next. If this is hard to follow, type in the formulas below and see if the practical application makes it clearer. Type column labels n in A6, X n in B6, then 0 in A7 and some initial state (eg., 0) in B7. Row 8 should contain =1+A7 in A8 and =LOOKUP(Random, OFFSET(CumMatrix,B7,0,1,5), StateVector) in B8. It is this last entry which is doing all the work. The OFFSET function directs the search to the appropriate row (B7 contains the row number, where the top row is always numbered 0) of the matrix CumMatrix; the parameters 1 and 5 refer to the number of rows (1) and the number of columns (5) which are available to be consulted. Once the function has found the appropriate entry it returns the corresponding value from StateVector. Stochastic Modelling 2003 Session 1, page 3

4 A8:B8 can be copied and pasted into as many rows as you like to simulate a sequence of consecutive values of the chain. Obtain a plot of the path of the Markov chain using the Line option of Chart Wizard. Give the column of observed values of the Markov chain (B7:Bwhatever) the name Chain. 7 Observed average discount We are going to find the observed proportion of time spent in states 0, 1, 2 and 3. Type 0 in F14, 1 in G14, 2 in H14, 3 in I14. Then in F15 enter the formula =COUNTIF(Chain,"=" & F14)/COUNT(Chain) to find the observed proportion of time spent in state 0. Copy the formula into G15:I15. Now think up some figures for the percentage discount at each discount level, starting from discount 0 in state 0, and enter them in F16:I16. To calculate the average discount achieved by the simulated policyholder over the period of the simulation, all we need is the SUMPRODUCT function, which essentially does a dot product of two vectors. Thus =SUMPRODUCT(F15:I15,F16:I16) gives the required answer for the observed average discount. 8 Array functions Excel has three built-in matrix functions, which can speed up a matrix-type calculation immensely but are a little tricky to type in. The functions are MDETERM, MINVERSE and MMULT. We are going to start off this section by using MMULT to produce successive calculations of the probability that the chain is in state i at time n starting from state 0. Theoretically, if v n is the vector of probabilities given by v n,i = P(X n =i), then v n satisfies the recurrence relation v n =v n-1 P. We are going to need to multiply a row vector v by the transition matrix P. Select the whole transition matrix (F3:I6) and give it a name, such as Transition_matrix. Now type in the initial vector of probabilities. Assuming that the motorist starts with no discount, the vector will be ( ): enter these four values into cells F18:I18. We have to multiply this by the matrix P to get the probabilities of being in the various states at the next time point. So select the cells F19:I19 to store the results of the calculation, type in the formula =MMULT(F18:I18,Transition_matrix) but DO NOT PRESS ENTER press Ctrl-Shift-Enter instead. All cells of the new array (F19:I19) will be filled with the same formula, shown inside braces (curly brackets). The Ctrl-Shift-Enter method is the way all arrays are entered. If you make a mistake in entering the formula you will have to clear the whole array at once, then reselect the whole area to have another go. Copy this formula down into several more blank rows by dragging the drag-handle at the bottom right corner of the selected area. Watch the probabilities converge to their long-run limits π. 9 Matrix inversion Let us try to automate the procedure used to calculate π. The first thing to bear in mind is that we need to solve πp = π, or in other words π(i P) = 0. To start with, then, let us get the identity matrix into the worksheet. Stochastic Modelling 2003 Session 1, page 4

5 Choose a 5 by 5 area such as L3:P7. Enter the names of the states (0, 1, 2, 3 in this case) into the first column (L3:L6) and the last row (M7:P7). We can fill the matrix using a single array function: select the matrix area M3:P6, type in the formula =IF(L3:L6=M7:P7,1,0) and press Ctrl-Shift-Enter. (This function returns a value of 1 if the row is equal to the column, 0 otherwise.) Immediately below this, in M8:P11, we can now enter the matrix I P: just use a simple subtraction formula like =M3-F3 and copy it across the whole of M8:P11. (Don't use Ctrl-Shift-Enter here as we will want to change the last column later on and Excel will not let you change just part of an array.) We know, however, that the matrix I P is not invertible, that there are multiple solutions to the equation π(i P) = 0, that the last equation in the set is redundant and that the solution we want has Σπ i = 1. This means that we need to replace the last column of the matrix I P by a column of 1's, resulting in a matrix M, say, then solving the equation πm = ( ), which is to say π = ( ) M 1. So you need to replace the last column of the matrix I P (P8:P11) by a column of 1's. Now select the region M13:P16, type in the formula =MINVERSE(M8:P11) and press Ctrl-Shift-Enter: as if by magic, the inverse of the matrix M appears in the selected area. You are probably expecting that we will now have to type in ( ) and perform a matrix multiplication to obtain the equilibrium probability vector π. In fact, though, a little thought will reveal that ( ) M 1 is simply the bottom row of the matrix M 1, so in fact the equilibrium distribution is now sitting in M16:P Multiple runs Suppose we are interested in a quantity whose distribution cannot easily be calculated, such as the position at time 50 of the random walk whose increments follow the general discrete distribution introduced in section 5. Even if we can't find the exact distribution by calculation, we can simulate the process a few times and calculate the sample mean, or draw a histogram of the results, or something similar. The Excel feature which we shall use is called a Data Table. Its purpose is to allow the user to perform a long calculation several times, using several different values for an input variable, without having to make a huge number of copies of the worksheet. In our case the input variable we want to change is the seed, and the output variable we want to study is the position of the RW at time 50. The one drawback of using the Data Table is that it requires both the input and output variables to be located on the same worksheet as the Data Table itself. So let us start by inserting a new worksheet to hold the data table, name it "DataTable", and write labels Input and Output near the top. The value in the Output cell must be equal to the position of the RW at time 50, so a formula like =SRW!E54 will do the job OK. The Input cell works the other way round: we need to ensure that a value entered in the Input cell is then copied across into the Seed cell on the Generator sheet. This involves replacing the Seed value with the formula =DataTable!B2, making sure that the Input cell contains a suitable value for the seed. Now set up the data table on the DataTable worksheet: in cells A5 to A20, to hold the different seeds, enter the formula =RAND(). Then in B4 use the formula =B1, so that the output value appears there. Next, select the range from A4 to B20, choose Data Table; leaving the Row Input Cell blank, enter $B$2 in the Column Input Cell and press Enter. Stochastic Modelling 2003 Session 1, page 5

6 The right-hand column of the table now fills with the values which the output cell would contain if the input cell contained the value shown in the left-hand column. Since the input cell contains only the seed for the random number generator, these are the output values for a number of independent runs of the simulation. You can construct a histogram or bar chart of the results, or investigate sample mean and variance. It goes without saying that a 'proper' simulation requires a good many more runs than 16, but naturally the data table can be made as large as you like, within the limits imposed by Excel (maximum is 65,000 or so). Once you have a collection of independently-generated output values you can analyse it in the way you would analyse any data set: find the sample mean and variance, plot a histogram of the output values, test for normality, etc. Stochastic Modelling 2003 Session 1, page 6