Why knitr? Very first steps in knitr Reviewing knitr ATM.Rnw Further exercises. PhD Workshop Series in

Transcription

1 PhD Workshop Series in Statistics and Applied Data Science (DSP.005) part two knitr Ass.-Prof. Dr. Arbeitsgruppe Stochastik/Statistik Fachbereich Mathematik Universität Salzburg Salzburg, December 2016

2 Aim of the game Starting point Very common situation (in research and in the economy): An elaborate report including graphics, tables and text has been produced (using Excel, Word, Powerpoint etc.) New data arrives, often on a weekly/monthly basis, or new data is added to an existing dataset The same report is needed for the new/updated data Do all calculations again, prepare all the tables and graphics again, and reproduce the report? Apart from wasting time and money, which other problems occur? Copy and paste is dangerous... Small (copy and paste) mistakes can have far-reaching consequences Can you assure to execute exactly the same manual data analysis steps as last time (Excel, etc.)? Reproducible results?

3 Aim of the game A possible solution - knitr Combine the power and flexibility of R with the typesetting capabilities of LaTeX R-Studio (user-interface for R) supports knitr R, R-Studio and LaTeX are freeware and run on all standard platforms! Let knitr do all the calculations, generate the graphics and tables and include numbers into the text Invest time in the first creation of the report - save time in all subsequent runs Use the saved time for more important and less annoying things than copy and paste (which robots can do much better than we)...

4 Very first steps in knitr Plan for today: Get knitr running on your laptops Run a minimal example Understand the basic building blocks of knitr Try to understand a pre-prepared knitr report Adjust/Manipulate/Extend the pre-prepared knitr report knitr has to be learned hands-on! Download and install the necessary software: (Basic) Miktex, Texmaker, R, R-Studio (in this order)

5 Very first steps in knitr Each knitr-file has the extension.rnw Each knitr-file consists of two building blocks: 1. LaTeX code 2. R-Code There are two types of R-Code: 1. chunks, i.e. code as separate paragraph (produce tables, graphics, etc.) Each chunk starts with <<some options>>= and ends 2. inline code, using the command Sexpr Knowing (basic) LaTeX and R is enough - the rest is combining blocks We start with a first minimal example demonstrating the LaTeX & R structure

6 Very first steps in knitr 1 \ d o c u m e n t c l a s s { a r t i c l e } 2 \ begin {document} <<h i s t o, f i g. width =10, f i g. h e i g h t =6, f i g. cap= Histogram, echo=false >>= 6 x< rnorm ( 1 0 0, 0, 1 ) 7 h i s t ( x, p r o b a b i l i t y = TRUE, c o l= l i g h t b l u e ) 9 10 The e s t i m a t e s f o r $\mu$ and $\ sigma $ a r e g i v e n by $\ o v e r l i n e {x} n=\ Sexpr {mean ( x ) }$ 11 and $ s ˆ2 n=\sexpr { sd ( x ) }$, whereby $n=\sexpr { l e n g t h ( x ) }$ \end{document} knitr mini.rnw Which part is LaTeX, which part is R?

7 Very first steps in knitr Download the file knitr mini.rnw from save it in a new folder, and open it with R-Studio A button Compile PDF appears (if not, something is not installed properly) Clicking it has the following effect knitr translates the R-Codes into standard LaTeX code and produces a.tex-file Plots produced are, by default, saved as pdfs in a folder named figure the.tex file is then compiled (pdflatex) and a final pdf is produced All files are saved in the folder where the knitr-file is located

8 Very first steps in knitr Histogram of x Density x Figure 1: Histogram The estimates for µ and σ are given by xn = and s 2 n = , whereby n = 100.

9 Very first steps in knitr Learning by doing - two exercises Exercise 1: Manipulate/Extend knitr mini.rnw in such a way that the resulting pdf looks like knitr mini extended.pdf Hint: par(mfrow = c(2,1)) can be used to have two plots in one graphic Exercise 2: Download the file knitr ATM.Rnw from save it in a new folder, and open it with R-Studio Install all R-packages loaded at the beginning of the first chunk and press Compile pdf Work through the.rnw and find out what the Code does Replace the ggplot-graphics by analogous graphics using the standard boxplot command Extend the report such that the output looks like knitr ATM extended.pdf

10 Reviewing knitr ATM.Rnw 1 %example l o a d s t h e ATM data s e t, a g g r e g a t e s and p r o d u c e s a s m a l l summary 2 %i n s t a l l R packages and ( b a s i c ) miktex f i r s t 3 4 %Block 1 : b a s i c LaTeX s e t t i n g s 5 6 \ d o c u m e n t c l a s s [ 1 2 pt ] { a r t i c l e } 7 \ usepackage {amsmath} 8 \ u sepackage { g r a p h i c x } 9 \ u sepackage { h y p e r r e f } 10 \ usepackage { eurosym } 11 \ u sepackage { c o l o r } 12 \ u sepackage { f l o a t } 13 \ s e t l e n g t h {\ t e x t h e i g h t }{650 pt } 14 \ s e t l e n g t h {\ t e x t w i d t h }{480 pt } 15 \ h o f f s e t = 15mm \ begin {document} knitr ATM/knitR ATM.Rnw

11 Reviewing knitr ATM.Rnw 1 2 %Block 2 : b a s i c R s e t u p : l o a d a l l r e q u i r e d packages 3 <<setup, i n c l u d e=false, cache=false>>= 4 l i b r a r y ( k n i t r ) 5 l i b r a r y ( g g p l o t 2 ) 6 l i b r a r y ( doby ) 7 l i b r a r y ( g r i d E x t r a ) 8 l i b r a r y ( RColorBrewer ) 9 l i b r a r y ( x t a b l e ) 10 Sys. s e t l o c a l e ( LC TIME, E n g l i s h ) #s e t E n g l i s h 11 # s e t g l o b a l chunk o p t i o n s 12 o p t s chunk $ s e t ( f i g. path= f i g u r e / g r a p h i c, f i g. a l i g n= c e n t e r, f i g. pos=! ht, echo=false, warning = FALSE) 13 #g l o b a l o p t i o n s f o r produced f i g u r e s ( name o f f i g u r e s, a u t o m a t i c c e n t e r i n g, e t c. ) 14 #echo=false : don t i n c l u d e R Code i n output, 15 #warning=false : p r i n t warning i n c o n s o l e but not i n pdf 16 #f i g. pos =! ht : p l a c e i t h e r e i n t h e doc 17 a< Sys. time ( ) knitr ATM/knitR ATM.Rnw

12 Reviewing knitr ATM.Rnw 1 \ t i t l e {\ v s p a c e{ 4cm} ATM w i t h d r a w a l s \ f o o t n o t e { This r e p o r t was c r e a t e d on \ Sexpr {a}}} 2 \ a u t h o r {Wolfgang T r u t s c h n i g } 3 %The f o o t n o t e p r i n t s t h e a c u t a l time c a l c u l a t e d i n t h e chunk above 4 5 \ m a k e t i t l e 6 7 \ s e c t i o n { Quick o v e r v i e w } knitr ATM/knitR ATM.Rnw

13 Reviewing knitr ATM.Rnw 1 <<r e s u l t s = a s i s >>= 2 #Download ATM. t x t and i n c l u d e t a b l e o f f i r s t s i x rows i n o u t p u t 3 A< r e a d. t a b l e ( h t t p : //www. t r u t s c h n i g. n e t / D a t e n s a t z. t x t, head=true) 4 A$ymd< as. Date (A$ymd) 5 A$month< as. numeric ( s u b s t r (A$ymd, 6, 7 ) ) 6 A$ y e a r< s u b s t r (A$ymd, 1, 4 ) 7 beg< min (A$ymd) ; end< max (A$ymd) 8 mis< nrow ( s u b s e t (A, i s. na (A$sum out )==1)) 9 H< s u b s e t (A,A$ h o l i d a y ==1) 10 V< s u b s e t (A,A$ h o l i d a y ==0.5) 11 B< A [ 1 : 6, 1 : 5 ] 12 B$ymd< as. c h a r a c t e r (B$ymd) 13 p r i n t ( x t a b l e (B, l a b e l = taba, c a p t i o n= F i r s t s i x l i n e s o f t h e d a t a s e t ), s i z e = f o o t n o t e s i z e, i n c l u d e. rownames=false) 14 #NB: x t a b l e p r e p a r e s t a b l e o u t p ut f o r LaTeX knitr ATM/knitR ATM.Rnw

14 Reviewing knitr ATM.Rnw 1 The d a t a s e t ( s e e Table \ r e f { taba }) c o n t a i n s d a i l y withdrawn amounts i n t h e p e r i o d from \ Sexpr {beg} t i l l \ Sexpr {end} (\ Sexpr { mis } e n t r i e s a r e m i s s i n g ). 2 We e x p e c t weekdays and h o l i d a y s to have a s t r o n g i n f l u e n c e on withdrawn amounts and, a d d i t i o n a l l y, to s e e an impact o f t h e f i n a n c i a l c r i s i s s t a r t i n g w i t h autumn F i g u r e \ r e f { f i g : b o x p l o t }, F i g u r e \ r e f { f i g : b o x p l o t monthly } as w e l l as Table \ r e f { tabb } c o n f i r m t h i s s u s p i c i o n. knitr ATM/knitR ATM.Rnw

15 Reviewing knitr ATM.Rnw 1 <<b o x p l o t, f i g. width =13, f i g. h e i g h t =6, f i g. cap=p a s t e ( B o x p l o t p e r day o f week and year, t h e medians a r e a l s o p r i n t e d i n Table \\ r e f { tabb } )>>= 2 #s i z e o f t h e p l o t ( width and h e i g h t ) i n i n c h e s, NB: d e f a u l t o u t p u t width i s \ t e x t w i d t h 3 p < g g p l o t ( data=a, a e s ( x=f a c t o r ( nr weekday ), y=sum out, f i l l =f a c t o r ( nr weekday ) ) ) 4 p < p + geom b o x p l o t ( o u t l i e r. s i z e =0) 5 p < p + f a c e t wrap ( y e a r ) 6 p < p + x l a b ( weekday ) 7 p < p + s c a l e f i l l d i s c r e t e ( name = Weekday ) 8 p < p + geom p o i n t ( data=h, c o l o u r= r e d, s i z e =1.5) 9 p < p + geom p o i n t ( data=v, c o l o u r= b l u e, s i z e =1.5) 10 p < p + theme bw ( ) 11 p 12 #NB: l a b e l o f f i g u r e i s a u t o m a t i c a l l y g e n e r a t e d as f i g : b o x p l o t knitr ATM/knitR ATM.Rnw

16 Reviewing knitr ATM.Rnw 1 <<r e s u l t s= a s i s >>= 2 medna< f u n c t i o n ( x ) {median ( x [ i s. na ( x ) ==0])} 3 # l i t t l e f u n c t i o n c a l c u l a t i n g t h e median ( i g n o r i n g m i s s i n g v a l u e s ) 4 BB< summaryby ( data=a, sum out nr weekday, FUN=c ( medna ) ) 5 #c a l c u l a t e meadian withdrawn amount per weekday 6 WD< A [ 1 : 7, 2 : 3 ] 7 BB< merge (BB,WD) 8 names (BB) [ 2 ]< sum out 9 BB$sum out< round (BB$sum out ) knitr ATM/knitR ATM.Rnw

17 Reviewing knitr ATM.Rnw 1 C o n s i n d e r i n g a l l y e a r s t o g e t h e r t h e median withdrawn amount i s \ Sexpr {BB$sum out [ 1 ] } \ e u r o {} on Mondays, \ Sexpr {BB$sum out [ 2 ] } \ euro {} on Tuesdays, \ Sexpr {BB$sum out [ 3 ] } \ euro {} on Wednesdays, 2 \ Sexpr {BB$sum out [ 4 ] } \ e u r o {} on Thursdays, \ Sexpr {BB$sum out [ 5 ] } \ e u r o {} on F r i d a y s, \ Sexpr {BB$sum out [ 6 ] } \ e u r o {} on S a t u r d a y s, and \ Sexpr {BB$sum out [ 7 ] } \ e u r o {} on Sundays. knitr ATM/knitR ATM.Rnw

18 Reviewing knitr ATM.Rnw 1 <<r e s u l t s= a s i s >>= 2 AA< summaryby ( data=a, sum out year+nr weekday, FUN=c ( medna ) ) 3 WD< A [ 1 : 7, 2 : 3 ] 4 AA< merge (AA,WD) 5 AA< AA[ o r d e r (AA$ nr weekday,aa$ y e a r ), ] 6 names (AA) [ 3 ]< sum out 7 AA< s u b s e t (AA, s e l e c t=c ( weekday, year, sum out ) ) 8 p r i n t ( x t a b l e (AA, l a b e l= tabb, c a p t i o n= Median withdrawn amount p e r y e a r and day o f week ), s i z e= s c r i p t s i z e, i n c l u d e. rownames=false) knitr ATM/knitR ATM.Rnw

19 Reviewing knitr ATM.Rnw 1 <<b o x p l o t monthly, f i g. width =13, f i g. h e i g h t =6, f i g. cap= B o x p l o t p e r month, t h e impact o f t h e f i n a n c i a l c r i s i s s t a r t i n g w i t h autumn 2008 >>= 2 B< A 3 B$ y e a r< as. f a c t o r (A$ y e a r ) 4 B$month< as. f a c t o r (A$month ) 5 f a r b e n< c ( gray50, magenta, g r e e n ) 6 p < g g p l o t ( data=b, a e s ( x=month, y=sum out ) ) 7 p < p + geom b o x p l o t ( a e s ( f i l l =y e a r ), o u t l i e r. s i z e =0) 8 p < p + s c a l e f i l l manual ( v a l u e s=f a r b e n ) 9 p < p + geom j i t t e r ( c o l o u r= gray30 ) 10 p < p + theme bw ( ) 11 p \end{document} knitr ATM/knitR ATM.Rnw

20 A small simulation study using knitr: GPS bias? Ten students of geoinformatics want to test GPS-based distance measurements They (consecutively) record the GPS-coordinates of (the outer track of) the 100m starting line in an athletics stadium close by, then (consecutively) walk along the outer track till the finishing line, and again record the GPS-coordinates. Each of them repeats this procedure 50 times For each of the 500 pairs they calculate the distance in meters Given the sample size of n = 500 they expect the mean distance to be pretty close to 100m (why?). All the bigger the surprise when the mean distance turns out to be roughly 102m What went wrong - just bad luck?

21 A small simulation study using knitr: GPS bias? x y

22 A small simulation study using knitr: GPS bias? x y

23 A small simulation study using knitr: GPS bias? What went wrong - just bad luck? We answer the question by means of simulations W.l.o.g. we assume that the starting point S and the end point Z have the following exact coordinates: S = (0, 0), Z = (100, 0) S, Z will denote the measured coordinates; F = (X 1, Y 1 ) denotes the measurement error in S, G = (X 2, Y 2 ) the measurement error in Z. In other words S = S + (X 1, Y 1 ) = (X 1, Y 1 ) Z = Z + (X 2, Y 2 ) = (100 + X 2, Y 2 ) The measured distance therefore given by S Z 2 = (100 + X 2 X 1 ) 2 + (Y 2 Y 1 ) 2 To simplify matters we assume that the errors follow a normal distribution, i.e. X 1, X 2, Y 2, Y 2 N (0, σ 2 ). Consider the case σ 2 = 15

24 A small simulation study using knitr: GPS bias? We simulate (or more) distance measurements (maybe 500 was not big enough) Use the code R-Code GPS.R auf to do the simulations We really get 102,2 as mean distance, i.e. a bias of 2,2. Exercise 3: Extend knitr GPS.Rnw to a report like knitr GPS extended.rnw Include a possible explanation of the observed GPS-bias