EP219:Data Analysis And Interpretation

Transcription

1 EP219:Data Analysis And Interpretation Report: Week 1 Team Poisson ous 28 July 2017 to 4 August

2 Contents 1 Problem Statement 3 2 Python Program 3 3 Histograms 6 4 Inference 7 5 Team responsibilities 7 6 Website 8 2

3 1 Problem Statement Given the text file Complete TAVG daily.txt that contains the daily temperature data from This file shows the average land temperature fluctuations averaged over the entire earth. Using this data the following analysis has been done. Extract the full data set to a numpy array. To this array add a column which shows the actual temperature on that day. Make a histogram of the temperature distribution for the year Re- peat this for the year Make a histogram of the temperature distribution for the temperature on 1st and 15th day of every month for the year 1912 and Python Program Here is the code for plotting the required histograms. Parts of the code to append extra column to the data imported as a numpy array, has been taken from Prof. Rentala s example code. 1 import pandas a s pd 2 import numpy a s np 3 import m a t p l o t l i b. pyplot as p l t 4 5 #Code to get the number o f commented l i n e s b e f o r e the a c t u a l data s t a r t s 6 D a t a f i l e=open ( Complete TAVG daily. txt ) 7 Ncommentedlines=0 8 while D a t a f i l e. read ( 1 )== % : 9 D a t a f i l e. r e a d l i n e ( ) 10 Ncommentedlines=Ncommentedlines+1 11 D a t a f i l e. c l o s e ( ) #S t o r i n g the data in a dataframe df with no headers and whitespace as d e l i m i t e r s ; the f i r s t Nommentedlines rows are skipped 14 df = pd. r e a d t a b l e ( Complete TAVG daily. txt, delim whitespace= True, header=none, skiprows=ncommentedlines ) #Data i s now s t o r e d in an numpy array 17 rawdatatable=df. v a l u e s 3

4 18 19 #Function i s c r e a t e d that takes any number to number tempfix = lambda t : t #f u n c t i o n i s v e c t o r i z e d, so i t can be a p p l i e d to an array 23 vtempfix = np. v e c t o r i z e ( tempfix ) #Vectorized f u n c t i o n i s a p p l i e d to column corresponding to temperature anamolies 26 newcolumn = vtempfix ( rawdatatable [ :, 5 ] ) #new column i s converted i n t o a 2 dimensional array so that i t can be appended to o r i g i n a l data 29 reshapednewcolumn = newcolumn. reshape ( newcolumn. s i z e, 1 ) #b e t t e r data i s the o r i g i n a l data with an added column o f temperature a x i s=1 i m p l i e s data i s appended columnwise 32 b e t t e r d a t a = np. append ( rawdatatable, reshapednewcolumn, a x i s =1) #cutdata c o n t a i n s only those rows o f b e t t e r d a t a f o r which the year i s i 35 cutdata = lambda i : b e t t e r d a t a [ ( b e t t e r d a t a [:,1]== i ), : ] 36 #A f u n c t i o n i s d e f i n e d to p l o t histograms f o r temperature data 37 def HistTemp ( year, c o l o u r ) : 38 #tempdata c o n t a i n s the temperature data o f each day f o r which the year was year 39 tempdata=cutdata ( year ) [ :, 6 ] #to get the l i m i t s o f histogram we f i n d the minimum and maximum v a l u e s o f temperature 4 i s m u l t i p l i e d and d ivided to get f i r s t bin value to the accuracy o f xmin=i n t (4 min ( tempdata ) ) / xmax=i n t (4 max( tempdata ) ) / #Number o f bins i s c a l c u l a t e d to get the array o f bins 46 Numbins=((xmax xmin ) 4) #binarray i s the array which i s p a s s e s as a parameter to histogram f u n c t i o n 49 binarray=np. l i n s p a c e ( xmin, xmax,num=numbins ) #Histogram i s p l o t t e d by p l t. h i s t f u n c t i o n 52 p l t. h i s t ( tempdata, bins=binarray, f a c e c o l o r=colour, alpha =0.9, rwidth =0.95) 53 4

5 54 #a x i s are l a b e l l e d and t i t l e i s given 55 p l t. t i t l e ( Temperature D i s t r i b u t i o n f o r the Year +s t r ( year ) ) 56 p l t. x l a b e l ( Temperature in C e l s i u s ) 57 p l t. y l a b e l ( Number o f days ) 58 p l t. xlim ( [ xmin, xmax ] ) 59 p l t. x t i c k s ( np. arange ( xmin, xmax, ) ) #p l o t i s d i s p l a y e d 63 p l t. show ( ) #Function i s c a l l e d f o r both the years r e q u i r e d 66 HistTemp (1912, green ) 67 HistTemp (2012, blue ) 68 #Function i s c r e a t e d f o r p r i n t i n g only datas o f two s p e c i f i c dates o f a month, year i s the year, day1 and day2 are the days o f the month r e q u i r e d 69 def HistSpecificTemp ( year, day1, day2, c o l o u r ) : 70 tempdata=cutdata ( year ) 71 r e f d a t a=tempdata [ ( tempdata [:,3]== day1 )+(tempdata [:,3]== day2 ), : ] [ :, 6 ] 72 #to get the l i m i t s o f histogram we f i n d the minimum and maximum v a l u e s o f temperature, 4 i s m u l t i p l i e d and d ivided to get f i r s t bin value to the accuracy o f xmin=i n t (4 min ( r e f d a t a ) ) / xmax=i n t (4 max( r e f d a t a ) ) / #Number o f bins i s c a l c u l a t e d to get the array o f bins 77 Numbins=((xmax xmin ) 4) #binarray i s the array which i s p a s s e s as a parameter to histogram f u n c t i o n 80 binarray=np. l i n s p a c e ( xmin, xmax,num=numbins ) #Histogram i s p l o t t e d by p l t. h i s t f u n c t i o n 83 p l t. h i s t ( refdata, bins=binarray, f a c e c o l o r=colour, alpha =0.9, rwidth =0.95) #a x i s are l a b e l l e d and t i t l e i s given 86 p l t. t i t l e ( Refined Temperature D i s t r i b u t i o n f o r the Year +s t r ( year ) ) 87 p l t. x l a b e l ( Temperature in C e l s i u s ) 88 p l t. y l a b e l ( Number o f days ) 89 p l t. xlim ( [ xmin, xmax ] ) 90 p l t. x t i c k s ( np. arange ( xmin, xmax, ) ) 5

6 91 #p l o t i s d i s p l a y e d 92 p l t. show ( ) #Function i s c a l l e d f o r year 1912 and HistSpecificTemp ( , 1, 15, green ) 96 HistSpecificTemp ( , 1, 15, blue ) #End o f Code 3 Histograms The following are the histograms for the temperature distribution of the year 1912 and 2012 considering the daily temperatures. (a) Year 1912 (b) Year 2012 Figure 1: the temperature distribution considering the daily temperatures Following are the histograms for the temperature distribution of the year 1912 and 2012 considering the temperatures on the 1st and 15th day of every month. 6

7 (a) Year 1912 (b) Year 2012 Figure 2: the temperature distribution considering the temperatures on the 1st and 15th day of every month 4 Inference We expected that the refined temperature histogram, having lesser data points is more likely to miss out the data points that lie on the fringes. Thus we expected the range of the distribution to be smaller. From the Histograms plotted we infer that the spread of the temperature in the Histogram of Temperature distribution for entire year is same as the spread of the Histogram of Temperature distribution for first and the fifteenth day of each month for both years 1912 and Moreover we see the temperature value at which the maxima of the temperature distribution occurs is higher in 2012 than in We infer from this that in fact global warming is a real issue. 5 Team responsibilities Project Leader - Saipriya Satyajit Programmer - Keshav Janyani Web Manager - Ananay Garg Report Writers - Abhisek Sahu and Ashay Telang 7

8 6 Website The link to our website is Team Poisson-ous All the assignments will be uploaded on this site. 8