Hot spot Analysis with Clustering on NIJ Data Set The Data Set:

Transcription

1 Hot spot Analysis with Clustering on NIJ Data Set The Data Set: The data set consists of Category, Call groups, final case type, case desc, Date, X-coordinates, Y- coordinates and census-tract. X-coordinates and Y-coordinates describes longitudes and longitudes respectively and censustract describes the z-coordinate. Case desc describes case description.there are multiple kinds of cases like disturbance, assault, robbery, robbery with weapon, threat, drugs, liquor, prostitution gambling etc. There are two types of categories: Street crimes and final case type.

2 The data set is an excel file(.xlxs) and is shown above. Data Preprocessing : The Data set procured from the Portland District Police Website had some dirty data in the form of blank cells or blank spaces.due to the presence of leading white spaces being too high we use the visualbasic scriptto remove the Leading blank/white spaces.

3 One the White spaces are removed we import the file into the python IDE. We represent the excel file in a pandas dataframe. The reason the pandas dataframe was chosen and not the numpy array was due to the fact that pandas wraps around numpy and to handle heterogenous data the pandas dataframe is quite convenient.

4 ArcMAP ArcGIS is a geographic information system (GIS) for working with maps and geographic information. Used for creating and using maps, compiling geographic data, analyzing mapped information, sharing and discovering geographic information, using maps and geographic information in a range of applications, and managing geographic information in a database.

5 Thus we successfully import the file in a dataframe for futhur analysis. Later, the data set contains some columns that are of no use for this analysis.as a result we drop these columns and create a new frame with the required columns. We set the precision of the pandas dataframe to be displayed upto 4 decimal places.although the spyder IDE has a built in option to display the precision according to the user specified value but still it is just a addition step to view the data more clearly in a IDE different from spyder. The data from the columns : x_coordinate and y_ Coordinate is too high and thus for the ease of calculation we perform the Z-score normalization on

6 these values.these values represent the location coordinates for the criminal activities. Now once the normalization is complete we get the values as follows :

7 After normalization we compute the K-Means clustering on the obtained preprocessed and reduced data set.

8 Here, the test shown above is generated using the number of clusters = 18 and max iterations to be 100.The tests were ran for no of clusters in the range of 4 22 and the number of clusters 18 gave an representable and expected result. Once the clustering was done, the generated cluster frame is appended to the original data frame and a separate column named : cluster. Thus it provides us the with the idea as to which row belongs to which cluster.

9 Thus the resulting data frame is as follows : We the write the obtained result back to an excel file and name it : DataMod.xlsx.

10 Graphical Visualization : We represent the clusters visually in python using the seaborn scatterplot. First we need the packages. We utilize the seaborn scatterplot to represent the clusters visually. The plot is x-coordinate vs y_coordinate and the hue attribute being the cluster column. The marker style selected is B and the size selected is 60. The obtained scatterplot is as follows :

11 The above plot shows all the 18 obtained clusters with different color codes. The next visualization includes real time plotting the data on the Portland map.

12 Importing the file into Arcmap : Thus the python-based analysis is now complete and we now move forward to representing the obtained data on a real-time coordinate based map representation. Now first we need to establish a connection to the file. This is done via the catalog side bar. Later on we choose the Connect to Folder option. Once we specify the file directory, the file is ready to get imported inside the workspace.note : We import the actual Sheets inside the workbook as a layer inside arcmap.

13 But we have one restriction here. To successfully work on individual data columns inside the arcmap workspace, we need feature classes for our data sets. Feature classes basically provide every record inside the dataset with a unique ID called OBJECT ID. Without the OBJECT ID its not possible to work on a attribute level.our analysis involves representing the generated clusters onto the map and cluster being an attribute in our dataset, adding the feature classes is an essential step.

14 Once we select the Create Feature Class option, we need to specify the X-coordinate, Y-coordinate and Z- coordinate to specify the X-Y values for our plot.our dataset has these values inside the columns : x_coordinate,y_coordinate and census_tract

15 respectively.we simple select the columns accordingly and complete the feature class creation process. Also, We specify a coordinate system for the class. We have the data from Portland North Oregon and this we select the co-ordinate system appropriately. We get the co-ordinate system information from the original portland shape file obtained from the website.

16 Thus as a result we convert the class co-ordinate plane into a plane of our choice. Once we complete the process we generate the required shape file in the specified

17 directory. Later on we import the shape file inside the workspace by a simple drag and drop operation. Once we finish the import, we get a general view as follows: The above representation just displays the points according to the coordinates specified inside the feature class. To represent the individual clusters we need to select the features by attributes.

18 Thus the attribute selection window, allows us to select the attribute of our choice by simply generating the queries.

19

20 Thus as seen, we are interested in the cluster attribute. We have a total of 18 clusters and the unique values are displayed to the right. To simply add each and every cluster as a layer in the arcmap we change the query value from 0 to and so on upto 17. As a result each and every cluster is now a separate layer as shown inside the Table of Contents pane to the left. Each point belonging to a particular cluster has a different color-code. The main step of geo-plotting the obtained points on a definite coordinate system is finished. Next step involves importing the required shape files for more detailed representation.

21 The shape files are: 1. Portland district shape file 2. Park trails. 3. Water banks. Thus the final representation is as follows : The above map gives an overall representation.

22 The above image highlights the area with high crime saturation.the above maps are generated with cluster size =4. For a more detailed description and analyzing the highly saturated criminal activity areas we analyze the data for no. of clusters = 18. We do the Hot-Spot Analysis. For the hot-spot analsys to work we need a fishnet grid shown as follows :

23 The above grid is a 30x30 grid superimposed onto the map. The Hot-spot analysis determines the area with the largest saturation of criminal activity. The analysis performed is the : Optimum hotspot analysis. It identifies statistically significant spatial clusters of high values (hot spots) and low values (cold spots).thus it segregates them on the basis of hot spots and cold-spots respectively. The hot-spots are marked with the red blocks on the fishnet grid. Along with the spot identification, we also find a confidence level for

24 each block of the grid.the confidence level of 99% is considered as the perfect hot-spot for our analysis. Thus the above representation displays the hot-spots with red colored blocks.