Data Analysis, Statistics, Machine Learning
|
|
- Ada Harmon
- 5 years ago
- Views:
Transcription
1 Data Analysis, Statistics, Machine Learning Leland Wilkinsn Adjunct Prfessr UIC Cmputer Science Chief Scien<st H2O.ai
2 Mst sta<s<cal predic<n mdels take ne f tw frms y = Σ j (β j x j ) + ε (addi<ve func<n) y = f(x j, ε) (nnlinear func<n) The dis<nc<n is imprtant The first frm is called an addi<ve mdel The secnd frm is called a nnlinear mdel Addi<ve mdels can be curvilinear (if terms are nnlinear) Nnlinear mdels cannt be transfrmed t linear Examples f linear r linearizable mdels are y =β 0 + β 1 x β p x p + ε y =αe βx+ ε Examples f nnlinear mdels are y =β 1 x 1 / β 2 x 2 + ε y = lgβ 1 x 1 ε 2 Cpyright 2016 Leland Wilkinsn
3 Regressin predicts a set f values n a variable y frm values n ne r mre x variables. This is dne by fivng a mathema<cal func<n that, fr any value(s) n the x variable(s), yields the mst prbable value f y. Simple linear mdel is y i = β 0 + β 1 x i + i Es<mates are ŷ i = ˆβ 0 + ˆβ 1 x i The badness r lss f this predic<n in a sample f values is represented by the discrepancies between the y values and their crrespnding predicted values. lss = n (y ŷ) 2 i=1 3 Cpyright 2016 Leland Wilkinsn
4 Ordinary Least Squares (OLS) Legendre (1805) Of all the principles that can be prpsed fr this purpse, I think there is nne mre general, mre exact, r easier t apply, than that which we have used in this wrk; it cnsists f making the sum f the squares f the errrs a minimum. By this methd, a kind f equilibrium is established amng the errrs which, since it prevents the extremes frm dmina<ng, is apprpriate fr revealing the state f the system which mst nearly appraches the truth. (transla<n by S<gler, 1986) 4 Cpyright 2016 Leland Wilkinsn
5 Regressin Francis Galtn (1887) He n<ced that the blue lines were the same length as the red nes That is, the line best predic<ng Y frm X regressed away frm the majr axis 5 Cpyright 2016 Leland Wilkinsn
6 Regressin Francis Galtn (1887) His data weren t as clean and linear as he imagined, but that didn t mager Height f Mid-Parent in Inches Height f Child in Inches Wachsmuth, Wilkinsn & Dallal, Cpyright 2016 Leland Wilkinsn
7 Regressin Francis Galtn (1887) That s because he aggregated ver different surces 80 Height f Father Height f Mther Height f Sn Height f Daughter Wachsmuth, Wilkinsn & Dallal, Cpyright 2016 Leland Wilkinsn
8 Es<ma<ng via Ordinary Least Squares 8 Cpyright 2016 Leland Wilkinsn
9 Es<ma<ng via Ordinary Least Squares Fr intercept (b 0 ) and slpe (b 1 ), we culd use calculus The way we did when we used maximum likelihd t es<mate mean and sd In this case, we want t minimize the sum f squared residuals SSE Given y i = b 0 b 1 x i + e i We sum the e i t get SSE SSE = n (y i b 0 b 1 x i ) 2 i=1 Cmpute the par<al deriva<ves with respect t b 0 and b 1 Set these deriva<ves t zer (where the minimum SSE exists) Slve the resul<ng simultaneus equa<ns This is what Legendre riginally did But there is an easier way 9 Cpyright 2016 Leland Wilkinsn
10 Es<ma<ng via OLS (we ll use matrices) Y = XB + E XB E X Y = X XB + X E (X X) 1 X Y =(X X) 1 (X X)B (X X) 1 X Y = B E = Y XB y Y = X = B = E = x 10 Cpyright 2016 Leland Wilkinsn
11 Es<ma<ng the regressin mdel parameters Hw d we knw we minimized the errr sum f squares? X Y = XY cs θ cs θ = X Y XY Pearsn crrela<n cefficient (if data are centered) length f E = E E Y Y and X are fixed (because f θ and X and Y ) Shrtest distance frm pint Y t line X is E QED θ XB X 11 Cpyright 2016 Leland Wilkinsn
12 Es<ma<n (assuming we sampled frm a ppula<n) B = ˆβ0 ˆβ 1 is a least- squares es<matr unbiased (assuming residuals are hmgeneus) smallest variance amng unbiased es<matrs Best Linear Unbiased Es<matr (BLUE) If the residuals are nrmally distributed B is a maximum likelihd es<matr We can d classical sta<s<cal tests n es<mates f parameters 12 Cpyright 2016 Leland Wilkinsn
13 Es<ma<ng via OLS Tw predictrs (same frmulas) Y = XB + E XB E X Y = X XB + X E (X X) 1 X Y =(X X) 1 (X X)B (X X) 1 X Y = B E = Y XB Y = X = B = E = Cpyright 2016 Leland Wilkinsn
14 The tw- predictr vectr space Y = XB + E Wikipedia 14 Cpyright 2016 Leland Wilkinsn
15 The tw- predictr vectr space It is pssible fr y nt t be crrelated much with either X 1 r X 2 yet be highly crrelated with the linear cmbina<n f X 1 and X 2 S dn t thrw ut predictrs by lking at their crrela<ns with the dependent variable X 1 E θ Y XB X 2 15 Cpyright 2016 Leland Wilkinsn
16 The tw- predictr vectr space Z is nt significantly related t X Z is nt significantly related t Y But the mul<ple crrela<n f Z with X and Y is almst 1! Z 0.00 Z X Y 16 Cpyright 2016 Leland Wilkinsn
17 Standardized Regressin Cefficients (Beta Weights) Standardize variables, then cmpute regressin Remves scales frm cnsidera<n f size f cefficients Scial scien<sts lve this stuff Why then are crrela<n cefficients s agrac<ve? Only bad reasns seem t cme t mind. Wrst f all, prbably, is the absence f any need t think abut units fr either variable. Given tw perfectly meaningless variables, ne is reminded f their meaninglessness when a regressin cefficient is given, since ne wnders hw t interpret its value. Tukey (1969) 17 Cpyright 2016 Leland Wilkinsn
18 Sums f Squares SSR = n (ŷ i ȳ i ) 2 i=1 regressin sum f squares (explained) SSE = n (y i ŷ i ) 2 i=1 errr sum f squares (unexplained) SST = n (y i ȳ i ) 2 i=1 ttal sum f squares 18 Cpyright 2016 Leland Wilkinsn
19 Gdness f fit Pearsn crrela<n ρ X,Y = ˆρ X,Y = r X,Y = COV (X, Y ) V AR(X)V AR(Y ) X Y XY (if X and Y are centered) Mul<ple crrela<n (sqrt f cefficient f determina<n) R 2 = n i=1 (ŷ i ȳ i ) 2 n i=1 (y i ȳ i ) 2 regressin sum f squares / ttal sum f squares R xx = 1 r x1 x 2 r x1 x p r x2 x 1 1 r x2 x p.... r xp x 1 r xp x 2 1 R yx = r yx1 r yx2 r yxp R 2 = R yx Rxx 1 R xy sum f squared crrela<ns with Y / crrela<ns amng X 19 Cpyright 2016 Leland Wilkinsn
20 Inference F- sta<s<c (test f significance fr verall predic<n) F p,n p 1 = R 2 /p (1 R 2 )/(n p 1) Cnfidence intervals n regressin cefficients c j = diag(x X) 1 j SSE s = n p 1 CI =(ˆβ ± t α/2 n p 1 s c j ) 20 Cpyright 2016 Leland Wilkinsn
21 An interes<ng applica<n f Ordinary Least Squares We have Z n x p : n fixed pints in p dimensins (cell twers, MDS r ther cnfigura<n, ) d n : a new pint y s es<mated distance t each pint in Z Slve fr vectr b p specifying lca<n f new pint z 1 z 2 z 7 y z 3 z 6 z 4 z 5 21 Cpyright 2016 Leland Wilkinsn
22 An interes<ng applica<n f Ordinary Least Squares We have Z n x p : n fixed pints in p dimensins (cell twers, MDS r ther cnfigura<n, ) d n : a new pint y s es<mated distance t each pint in Z Slve fr vectr b p specifying lca<n f new pint d Z X y i j m s = n i=2 i 1 j=1 k=1 p (Z ik Z jk ) 2 (a scaling cnstant) y m = d 2 i d 2 j + s, j =1,,i 1, i =2,,n, m=(i 1)(i 2)/2+j X mk = 2(Z ik Z jk ), j =1,,i 1, i =2,,n, m=(i 1)(i 2)/2+j b =(X X) 1 X y 22 Cpyright 2016 Leland Wilkinsn
23 OLS Assump<ns The true mdel is linear in the parameters The X variables are nt randm and are measured withut errr The X variables are nt cllinear Residuals are uncrrelated/independent The residuals have cnstant variance (hmscedas<city) If t and F test sta<s<cs are cmputed Residuals must be Nrmally distributed Randm sample frm a ppula<n 23 Cpyright 2016 Leland Wilkinsn
24 Evalua<ng OLS assump<ns The true mdel is linear in the parameters Use LOESS t put a smth thrugh residual plt 24 Cpyright 2016 Leland Wilkinsn
25 Transfrma<ns Dealing with nnlinearity Wilkinsn, Blank, & Gruber (1996) 25 Cpyright 2016 Leland Wilkinsn
26 Transfrma<ns Dealing with nnlinearity Tukey- Msteller bulging rule Ascend y ladder (p > 1) x = x p y = y p Descend x ladder (p < 1) Ascend x ladder (p > 1) Descend y ladder (p < 1) 26 Cpyright 2016 Leland Wilkinsn
27 Evalua<ng OLS assump<ns The X variables are nt randm and are measured withut errr There is n simple test r graphic fr this Knw the surce f yur measurements If there is measurement errr, yu can use errrs- in- the- variables methds Or, yu can just frgeddabu<t, which is what mst f the wrld des Wh cares if yur cefficient es<mates are biased? They are usually biased dwnward, s n harm dne Just dn t shw them t a scial scien<st Scial scien<sts lve latent variable mdels They are Platnists 27 Cpyright 2016 Leland Wilkinsn
28 Evalua<ng OLS assump<ns The X variables are nt cllinear Values f Variance Infla<n Factr (VIF) abve 10 are wrrisme VIF = 1 1 R 2 i R 2 =.995 R 2 = Cpyright 2016 Leland Wilkinsn
29 Evalua<ng OLS assump<ns Residuals are uncrrelated/independent An ACF plt can help spt vila<ns, but there are ther types f dependencies Ecnmists spend all their <me wrrying abut this And fr gd reasn serial dependence is mre txic than utliers Dn t even THINK f using rdinary linear regressin (trend line) n <me series data 29 Cpyright 2016 Leland Wilkinsn
30 Evalua<ng OLS assump<ns The residuals have cnstant variance (hmscedas<city) Try transfrma<ns (usually lgging wrks fr a pwer mdel) Or use ne f the heterscedas<city crrec<ns (MacKinnn- White, etc.) 30 Cpyright 2016 Leland Wilkinsn
31 Transfrma<ns Dealing with heterscedas<city lg- lg transfrma<n 31 Cpyright 2016 Leland Wilkinsn
32 Evalua<ng OLS assump<ns If t and F test sta<s<cs are cmputed Residuals must be Nrmally distributed Randm sample frm a ppula<n Frget abut tests fr nrmality; they are wrthless 32 Cpyright 2016 Leland Wilkinsn
33 Evalua<ng OLS assump<ns If t and F test sta<s<cs are cmputed Residuals must be Nrmally distributed 33 Cpyright 2016 Leland Wilkinsn
34 Evalua<ng OLS assump<ns Leverage Diagnal f the hat matrix (puts the hat n the parameters) H = X(X X) 1 X Ck s D Measures the decrement in predic<n by remving an bserva<n A type f leverage measure that is transfrmable t an apprximate F 34 Cpyright 2016 Leland Wilkinsn
35 Evalua<ng OLS assump<ns Leverage 35 Cpyright 2016 Leland Wilkinsn
36 Mdel Selec<n Par<al Residual Plts Each plt cnsists f sets f residuals plged against each ther. One set, n the ver<cal axis, cnsists f the residuals frm regressing y n all the X (predictr) variables except fr the predictr n the hrizntal axis. The ther set, n the hrizntal axis, cnsists f the residuals frm regressing X i n all the ther X variables. The result is a plt which shws yu hw each X variable is related t the Y variable when all the ther X variables are taken int accunt. Par<al residual plts have several useful features: The slpe f the line in the plt is the par<al regressin regressin cefficient crrespnding t the predictr in the plt. A line with a steep slpe and nice lking residuals is a sign that a predictr belngs in the mdel. The residuals frm this line are the same as the residuals frm regressing Y n all the predictrs. The plt helps yu t judge whether the cndi<nal rela<nship is linear r nnlinear. Extreme values n the hrizntal axis help yu t iden<fy high leverage pints. 36 Cpyright 2016 Leland Wilkinsn
37 Mdel Selec<n Par<al Residual Plts time = distance climb 37 Cpyright 2016 Leland Wilkinsn
38 Mdel Selec<n Frward Selec<n Regressin Backward Elimina<n Regressin Stepwise Regressin All Pssible Subsets Regressin 38 Cpyright 2016 Leland Wilkinsn
39 Mdel Selec<n Myers, J.L., Well, A.D., Lrch Jr, R.F. (2013). Research Design and Sta<s<cal Analysis (3 rd Ed.). Rutledge. 39 Cpyright 2016 Leland Wilkinsn
40 Mdel Selec<n Akaike Infrma<n Criterin (AIC) Lngley Data: Predic<ng TOTAL (RSQ =.995) 40 Cpyright 2016 Leland Wilkinsn
41 Mdel Selec<n (m submdels) AIC = 2(p + 2) 2 lg(l) AIC k = AIC k AIC min L k =exp( AIC/2) W k = L k / m i=1 L i rela<ve AIC rela<ve likelihd AIC weights RSQ Increment AIC Weight 41 Cpyright 2016 Leland Wilkinsn
42 Mdel Selec<n (m submdels) 42 Cpyright 2016 Leland Wilkinsn
43 Other appraches t mul<cllinearity Regulariza<n Ridge regressin (Herl, 1962) Minimize n (y i x i β) 2 + λ i=1 p j=1 β 2 j S, ˆβ =(X X + λi) 1 X y We inflate the diagnal t reduce the influence f the ff- diagnal cvariances Ridge regressin shrinks the es<mates tward zer, intrducing bias But this reduces the variance f the es<mates 43 Cpyright 2016 Leland Wilkinsn
44 Other appraches t mul<cllinearity Regulariza<n LASSO (Least Abslute Value Shrinkage and Selec<n Operatr) Minimize Slu<n requires itera<n n (y i x i β) 2 + λ i=1 Allws sme cefficients t g t zer with thers nnzer Efrn and Tibshirani use Least Angle Regressin with Shrinkage (LARS) Similar t stepwise regressin p β j j=1 44 Cpyright 2016 Leland Wilkinsn
45 Lgis<c Regressin OLS es<mates 2 1 PHD GRE 45 Cpyright 2016 Leland Wilkinsn
46 Lgis<c Regressin LR es<mates 2 1 PHD GRE 46 Cpyright 2016 Leland Wilkinsn
47 Lgis<c Regressin Mdel and es<ma<n p(x; β) = lgit(p) = lg L(β; x 1,...,x n )= l(β; x 1,...,x n )= exβ 1+e xβ = 1 1+e xβ p(x) 1 p(x) = xβ n i=1 p y i i (1 p i) 1 y i n lgit i lg(1 + p i ) i=1 Must use itera<ve p<miza<n t find maximum Different mdel fr mre than tw categries x = x 0,,x p β = β 0,, β p lg- dds likelihd (Binmial) lg- likelihd 47 Cpyright 2016 Leland Wilkinsn
48 Pissn Regressin (lg- linear mdel) Mdel and es<ma<n E[Y x] =e xβ Pissn distribu<n has nly 1 parameter p(y; x, β) = eyxβ e exβ y! L(β; y, x 1,...,x n )= l(β; y, x 1,...,x n )= n e y ix i β e ex i β y i! n {(y i x i β) e xiβ lg(y i!)} i=1 i=1 y is integer valued likelihd lg- likelihd (dn t need the yellw term In rder t maximize) Must use itera<ve p<miza<n t find maximum 48 Cpyright 2016 Leland Wilkinsn
49 Pissn Regressin OLS Pissn 49 Cpyright 2016 Leland Wilkinsn
50 Generalized Linear Mdels (GLM) Nelder & Wedderburn (1972) Nt t be cnfused with General Linear Mdel (GLM) fr OLS with r withut dummy variables Generalized Least Squares (GLS) fr dealing with heterscedas<city Es<ma<n dne thrugh Itera<vely Reweighted Least Squares (IRWLS) xβ = g(e[y ]) E[Y ]=µ = g 1 (xβ) link func<n mdeling mean f Y thrugh inverse f link func<n I Distribu<n Link Func<n Name Link Func<n Nrmal Binmial Pissn Iden<ty Lgit Lg xβ = µ µ xβ = lg 1 µ xβ = lg(µ) 50 Cpyright 2016 Leland Wilkinsn
51 Nnlinear Regressin Func<ns f the frm y = f(x + ε) Le} is linearizable by transfrma<n, right is intrinsically nnlinear y = e β 0+β 1 x+ y = e β 0+β 1 x + 51 Cpyright 2016 Leland Wilkinsn
52 Nnlinear Regressin a+b/x+c lg(x)) E[Y ]=e Minimize SSE (Newtn, Quasi- Newtn, Metrplis, ) Magic, right? 52 Cpyright 2016 Leland Wilkinsn
53 Nnlinear Regressin Things t wrry abut Dn t waste yur <me with R 2 There are all srts f defini<ns All are nnsense The bad nes are ridiculusly large Lk at sum f squared errrs instead Dn t hunt arund fr nnlinear equa<ns that wrk That s a fishing expedi<n The equa<n yu chse shuld be driven by thery and the dmain it applies t Half the <me yur itera<ve fivng methd will crak That s a sign that there s a lcal minimum Or yur equa<n is wrng Or yur star<ng values are wildly ff- target Dn t waste yur <me lking fr anther cmputer prgram Nnlinear prgrams are ntriusly finicky Yur data are prbably crap r yur mdel is ridiculus Lk at the fit befre yu examine any sta<s<cs Mst f the <me yu have ne predictr and ne dependent variable S lk at the figed equa<n Ignre the fit sta<s<cs and tests f significance un<l it lks gd EVERYTHING fits Keep in mind that almst any bad mdel will lk pregy gd If it desn t make there<cal sense, dn t trust it 53 Cpyright 2016 Leland Wilkinsn
54 References McCullagh, P. and Nelder, J. (1989). Generalized Linear Mdels, Secnd Editin. Bca Ratn: Chapman and HalłCRC. Tukey, J.W. (1969). Analyzing data: Sanctificatin r detective wrk? American Psychlgist, 24, Cpyright 2016 Leland Wilkinsn
Data Analysis, Statistics, Machine Learning
Data Analysis, Statistics, Machine Learning Leland Wilkinsn Adjunct Prfessr UIC Cmputer Science Chief Scien
More informationData Analysis, Statistics, Machine Learning
Data Analysis, Statistics, Machine Learning Leland Wilkinsn Adjunct Prfessr UIC Cmputer Science Chief Scien
More informationResampling Methods. Cross-validation, Bootstrapping. Marek Petrik 2/21/2017
Resampling Methds Crss-validatin, Btstrapping Marek Petrik 2/21/2017 Sme f the figures in this presentatin are taken frm An Intrductin t Statistical Learning, with applicatins in R (Springer, 2013) with
More informationSimple Linear Regression (single variable)
Simple Linear Regressin (single variable) Intrductin t Machine Learning Marek Petrik January 31, 2017 Sme f the figures in this presentatin are taken frm An Intrductin t Statistical Learning, with applicatins
More information4th Indian Institute of Astrophysics - PennState Astrostatistics School July, 2013 Vainu Bappu Observatory, Kavalur. Correlation and Regression
4th Indian Institute f Astrphysics - PennState Astrstatistics Schl July, 2013 Vainu Bappu Observatry, Kavalur Crrelatin and Regressin Rahul Ry Indian Statistical Institute, Delhi. Crrelatin Cnsider a tw
More informationA Matrix Representation of Panel Data
web Extensin 6 Appendix 6.A A Matrix Representatin f Panel Data Panel data mdels cme in tw brad varieties, distinct intercept DGPs and errr cmpnent DGPs. his appendix presents matrix algebra representatins
More information[COLLEGE ALGEBRA EXAM I REVIEW TOPICS] ( u s e t h i s t o m a k e s u r e y o u a r e r e a d y )
(Abut the final) [COLLEGE ALGEBRA EXAM I REVIEW TOPICS] ( u s e t h i s t m a k e s u r e y u a r e r e a d y ) The department writes the final exam s I dn't really knw what's n it and I can't very well
More informationCHAPTER 4 DIAGNOSTICS FOR INFLUENTIAL OBSERVATIONS
CHAPTER 4 DIAGNOSTICS FOR INFLUENTIAL OBSERVATIONS 1 Influential bservatins are bservatins whse presence in the data can have a distrting effect n the parameter estimates and pssibly the entire analysis,
More informationCOMP 551 Applied Machine Learning Lecture 5: Generative models for linear classification
COMP 551 Applied Machine Learning Lecture 5: Generative mdels fr linear classificatin Instructr: Herke van Hf (herke.vanhf@mail.mcgill.ca) Slides mstly by: Jelle Pineau Class web page: www.cs.mcgill.ca/~hvanh2/cmp551
More informationInternal vs. external validity. External validity. This section is based on Stock and Watson s Chapter 9.
Sectin 7 Mdel Assessment This sectin is based n Stck and Watsn s Chapter 9. Internal vs. external validity Internal validity refers t whether the analysis is valid fr the ppulatin and sample being studied.
More informationMidwest Big Data Summer School: Machine Learning I: Introduction. Kris De Brabanter
Midwest Big Data Summer Schl: Machine Learning I: Intrductin Kris De Brabanter kbrabant@iastate.edu Iwa State University Department f Statistics Department f Cmputer Science June 24, 2016 1/24 Outline
More informationResampling Methods. Chapter 5. Chapter 5 1 / 52
Resampling Methds Chapter 5 Chapter 5 1 / 52 1 51 Validatin set apprach 2 52 Crss validatin 3 53 Btstrap Chapter 5 2 / 52 Abut Resampling An imprtant statistical tl Pretending the data as ppulatin and
More informationCHAPTER 24: INFERENCE IN REGRESSION. Chapter 24: Make inferences about the population from which the sample data came.
MATH 1342 Ch. 24 April 25 and 27, 2013 Page 1 f 5 CHAPTER 24: INFERENCE IN REGRESSION Chapters 4 and 5: Relatinships between tw quantitative variables. Be able t Make a graph (scatterplt) Summarize the
More informationPSU GISPOPSCI June 2011 Ordinary Least Squares & Spatial Linear Regression in GeoDa
There are tw parts t this lab. The first is intended t demnstrate hw t request and interpret the spatial diagnstics f a standard OLS regressin mdel using GeDa. The diagnstics prvide infrmatin abut the
More information3.4 Shrinkage Methods Prostate Cancer Data Example (Continued) Ridge Regression
3.3.4 Prstate Cancer Data Example (Cntinued) 3.4 Shrinkage Methds 61 Table 3.3 shws the cefficients frm a number f different selectin and shrinkage methds. They are best-subset selectin using an all-subsets
More informationInference in the Multiple-Regression
Sectin 5 Mdel Inference in the Multiple-Regressin Kinds f hypthesis tests in a multiple regressin There are several distinct kinds f hypthesis tests we can run in a multiple regressin. Suppse that amng
More informationCS 477/677 Analysis of Algorithms Fall 2007 Dr. George Bebis Course Project Due Date: 11/29/2007
CS 477/677 Analysis f Algrithms Fall 2007 Dr. Gerge Bebis Curse Prject Due Date: 11/29/2007 Part1: Cmparisn f Srting Algrithms (70% f the prject grade) The bjective f the first part f the assignment is
More informationx 1 Outline IAML: Logistic Regression Decision Boundaries Example Data
Outline IAML: Lgistic Regressin Charles Suttn and Victr Lavrenk Schl f Infrmatics Semester Lgistic functin Lgistic regressin Learning lgistic regressin Optimizatin The pwer f nn-linear basis functins Least-squares
More informationChapter 3: Cluster Analysis
Chapter 3: Cluster Analysis } 3.1 Basic Cncepts f Clustering 3.1.1 Cluster Analysis 3.1. Clustering Categries } 3. Partitining Methds 3..1 The principle 3.. K-Means Methd 3..3 K-Medids Methd 3..4 CLARA
More informationFall 2013 Physics 172 Recitation 3 Momentum and Springs
Fall 03 Physics 7 Recitatin 3 Mmentum and Springs Purpse: The purpse f this recitatin is t give yu experience wrking with mmentum and the mmentum update frmula. Readings: Chapter.3-.5 Learning Objectives:.3.
More informationIntroduction to Regression
Intrductin t Regressin Administrivia Hmewrk 6 psted later tnight. Due Friday after Break. 2 Statistical Mdeling Thus far we ve talked abut Descriptive Statistics: This is the way my sample is Inferential
More informationLecture 2: Supervised vs. unsupervised learning, bias-variance tradeoff
Lecture 2: Supervised vs. unsupervised learning, bias-variance tradeff Reading: Chapter 2 STATS 202: Data mining and analysis September 27, 2017 1 / 20 Supervised vs. unsupervised learning In unsupervised
More informationAP Physics Kinematic Wrap Up
AP Physics Kinematic Wrap Up S what d yu need t knw abut this mtin in tw-dimensin stuff t get a gd scre n the ld AP Physics Test? First ff, here are the equatins that yu ll have t wrk with: v v at x x
More informationLecture 2: Supervised vs. unsupervised learning, bias-variance tradeoff
Lecture 2: Supervised vs. unsupervised learning, bias-variance tradeff Reading: Chapter 2 STATS 202: Data mining and analysis September 27, 2017 1 / 20 Supervised vs. unsupervised learning In unsupervised
More informationComputational modeling techniques
Cmputatinal mdeling techniques Lecture 2: Mdeling change. In Petre Department f IT, Åb Akademi http://users.ab.fi/ipetre/cmpmd/ Cntent f the lecture Basic paradigm f mdeling change Examples Linear dynamical
More informationThe general linear model and Statistical Parametric Mapping I: Introduction to the GLM
The general linear mdel and Statistical Parametric Mapping I: Intrductin t the GLM Alexa Mrcm and Stefan Kiebel, Rik Hensn, Andrew Hlmes & J-B J Pline Overview Intrductin Essential cncepts Mdelling Design
More informationBootstrap Method > # Purpose: understand how bootstrap method works > obs=c(11.96, 5.03, 67.40, 16.07, 31.50, 7.73, 11.10, 22.38) > n=length(obs) >
Btstrap Methd > # Purpse: understand hw btstrap methd wrks > bs=c(11.96, 5.03, 67.40, 16.07, 31.50, 7.73, 11.10, 22.38) > n=length(bs) > mean(bs) [1] 21.64625 > # estimate f lambda > lambda = 1/mean(bs);
More informationSPH3U1 Lesson 06 Kinematics
PROJECTILE MOTION LEARNING GOALS Students will: Describe the mtin f an bject thrwn at arbitrary angles thrugh the air. Describe the hrizntal and vertical mtins f a prjectile. Slve prjectile mtin prblems.
More informationThis section is primarily focused on tools to aid us in finding roots/zeros/ -intercepts of polynomials. Essentially, our focus turns to solving.
Sectin 3.2: Many f yu WILL need t watch the crrespnding vides fr this sectin n MyOpenMath! This sectin is primarily fcused n tls t aid us in finding rts/zers/ -intercepts f plynmials. Essentially, ur fcus
More informationAP Statistics Practice Test Unit Three Exploring Relationships Between Variables. Name Period Date
AP Statistics Practice Test Unit Three Explring Relatinships Between Variables Name Perid Date True r False: 1. Crrelatin and regressin require explanatry and respnse variables. 1. 2. Every least squares
More informationSmoothing, penalized least squares and splines
Smthing, penalized least squares and splines Duglas Nychka, www.image.ucar.edu/~nychka Lcally weighted averages Penalized least squares smthers Prperties f smthers Splines and Reprducing Kernels The interplatin
More informationLHS Mathematics Department Honors Pre-Calculus Final Exam 2002 Answers
LHS Mathematics Department Hnrs Pre-alculus Final Eam nswers Part Shrt Prblems The table at the right gives the ppulatin f Massachusetts ver the past several decades Using an epnential mdel, predict the
More informationKinetic Model Completeness
5.68J/10.652J Spring 2003 Lecture Ntes Tuesday April 15, 2003 Kinetic Mdel Cmpleteness We say a chemical kinetic mdel is cmplete fr a particular reactin cnditin when it cntains all the species and reactins
More informationLecture 10, Principal Component Analysis
Principal Cmpnent Analysis Lecture 10, Principal Cmpnent Analysis Ha Helen Zhang Fall 2017 Ha Helen Zhang Lecture 10, Principal Cmpnent Analysis 1 / 16 Principal Cmpnent Analysis Lecture 10, Principal
More informationIAML: Support Vector Machines
1 / 22 IAML: Supprt Vectr Machines Charles Suttn and Victr Lavrenk Schl f Infrmatics Semester 1 2 / 22 Outline Separating hyperplane with maimum margin Nn-separable training data Epanding the input int
More informationOur Lady Star of the Sea Religious Education CIRCLE OF GRACE LESSON PLAN - Grade 1
Our Lady Star f the Sea Religius Educatin CIRCLE OF GRACE LESSON PLAN - Grade 1 Opening Prayer: (ech prayer) Hly Spirit (ech) Shw us the way (ech) Be with us in all we think.. d and say (ech) Amen GETTING
More informationComparing Several Means: ANOVA. Group Means and Grand Mean
STAT 511 ANOVA and Regressin 1 Cmparing Several Means: ANOVA Slide 1 Blue Lake snap beans were grwn in 12 pen-tp chambers which are subject t 4 treatments 3 each with O 3 and SO 2 present/absent. The ttal
More informationPreparation work for A2 Mathematics [2018]
Preparatin wrk fr A Mathematics [018] The wrk studied in Y1 will frm the fundatins n which will build upn in Year 13. It will nly be reviewed during Year 13, it will nt be retaught. This is t allw time
More information, which yields. where z1. and z2
The Gaussian r Nrmal PDF, Page 1 The Gaussian r Nrmal Prbability Density Functin Authr: Jhn M Cimbala, Penn State University Latest revisin: 11 September 13 The Gaussian r Nrmal Prbability Density Functin
More informationMODULE 1. e x + c. [You can t separate a demominator, but you can divide a single denominator into each numerator term] a + b a(a + b)+1 = a + b
. REVIEW OF SOME BASIC ALGEBRA MODULE () Slving Equatins Yu shuld be able t slve fr x: a + b = c a d + e x + c and get x = e(ba +) b(c a) d(ba +) c Cmmn mistakes and strategies:. a b + c a b + a c, but
More informationCOMP 551 Applied Machine Learning Lecture 4: Linear classification
COMP 551 Applied Machine Learning Lecture 4: Linear classificatin Instructr: Jelle Pineau (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/cmp551 Unless therwise nted, all material psted
More informationDifferentiation Applications 1: Related Rates
Differentiatin Applicatins 1: Related Rates 151 Differentiatin Applicatins 1: Related Rates Mdel 1: Sliding Ladder 10 ladder y 10 ladder 10 ladder A 10 ft ladder is leaning against a wall when the bttm
More informationCAUSAL INFERENCE. Technical Track Session I. Phillippe Leite. The World Bank
CAUSAL INFERENCE Technical Track Sessin I Phillippe Leite The Wrld Bank These slides were develped by Christel Vermeersch and mdified by Phillippe Leite fr the purpse f this wrkshp Plicy questins are causal
More informationINSTRUMENTAL VARIABLES
INSTRUMENTAL VARIABLES Technical Track Sessin IV Sergi Urzua University f Maryland Instrumental Variables and IE Tw main uses f IV in impact evaluatin: 1. Crrect fr difference between assignment f treatment
More informationTrigonometric Ratios Unit 5 Tentative TEST date
1 U n i t 5 11U Date: Name: Trignmetric Ratis Unit 5 Tentative TEST date Big idea/learning Gals In this unit yu will extend yur knwledge f SOH CAH TOA t wrk with btuse and reflex angles. This extensin
More informationCOMP 551 Applied Machine Learning Lecture 11: Support Vector Machines
COMP 551 Applied Machine Learning Lecture 11: Supprt Vectr Machines Instructr: (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/cmp551 Unless therwise nted, all material psted fr this curse
More informationPhysics 2010 Motion with Constant Acceleration Experiment 1
. Physics 00 Mtin with Cnstant Acceleratin Experiment In this lab, we will study the mtin f a glider as it accelerates dwnhill n a tilted air track. The glider is supprted ver the air track by a cushin
More informationMaximum A Posteriori (MAP) CS 109 Lecture 22 May 16th, 2016
Maximum A Psteriri (MAP) CS 109 Lecture 22 May 16th, 2016 Previusly in CS109 Game f Estimatrs Maximum Likelihd Nn spiler: this didn t happen Side Plt argmax argmax f lg Mther f ptimizatins? Reviving an
More informationLab 1 The Scientific Method
INTRODUCTION The fllwing labratry exercise is designed t give yu, the student, an pprtunity t explre unknwn systems, r universes, and hypthesize pssible rules which may gvern the behavir within them. Scientific
More informationIN a recent article, Geary [1972] discussed the merit of taking first differences
The Efficiency f Taking First Differences in Regressin Analysis: A Nte J. A. TILLMAN IN a recent article, Geary [1972] discussed the merit f taking first differences t deal with the prblems that trends
More informationWe can see from the graph above that the intersection is, i.e., [ ).
MTH 111 Cllege Algebra Lecture Ntes July 2, 2014 Functin Arithmetic: With nt t much difficulty, we ntice that inputs f functins are numbers, and utputs f functins are numbers. S whatever we can d with
More informationThermodynamics Partial Outline of Topics
Thermdynamics Partial Outline f Tpics I. The secnd law f thermdynamics addresses the issue f spntaneity and invlves a functin called entrpy (S): If a prcess is spntaneus, then Suniverse > 0 (2 nd Law!)
More informationModule 3: Gaussian Process Parameter Estimation, Prediction Uncertainty, and Diagnostics
Mdule 3: Gaussian Prcess Parameter Estimatin, Predictin Uncertainty, and Diagnstics Jerme Sacks and William J Welch Natinal Institute f Statistical Sciences and University f British Clumbia Adapted frm
More informationLab #3: Pendulum Period and Proportionalities
Physics 144 Chwdary Hw Things Wrk Spring 2006 Name: Partners Name(s): Intrductin Lab #3: Pendulum Perid and Prprtinalities Smetimes, it is useful t knw the dependence f ne quantity n anther, like hw the
More informationPreparation work for A2 Mathematics [2017]
Preparatin wrk fr A2 Mathematics [2017] The wrk studied in Y12 after the return frm study leave is frm the Cre 3 mdule f the A2 Mathematics curse. This wrk will nly be reviewed during Year 13, it will
More informationPattern Recognition 2014 Support Vector Machines
Pattern Recgnitin 2014 Supprt Vectr Machines Ad Feelders Universiteit Utrecht Ad Feelders ( Universiteit Utrecht ) Pattern Recgnitin 1 / 55 Overview 1 Separable Case 2 Kernel Functins 3 Allwing Errrs (Sft
More informationCOMP 551 Applied Machine Learning Lecture 9: Support Vector Machines (cont d)
COMP 551 Applied Machine Learning Lecture 9: Supprt Vectr Machines (cnt d) Instructr: Herke van Hf (herke.vanhf@mail.mcgill.ca) Slides mstly by: Class web page: www.cs.mcgill.ca/~hvanh2/cmp551 Unless therwise
More informationFinding the Earth s magnetic field
Labratry #6 Name: Phys 1402 - Dr. Cristian Bahrim Finding the Earth s magnetic field The thery accepted tday fr the rigin f the Earth s magnetic field is based n the mtin f the plasma (a miture f electrns
More informationThe Law of Total Probability, Bayes Rule, and Random Variables (Oh My!)
The Law f Ttal Prbability, Bayes Rule, and Randm Variables (Oh My!) Administrivia Hmewrk 2 is psted and is due tw Friday s frm nw If yu didn t start early last time, please d s this time. Gd Milestnes:
More informationData Analysis, Statistics, Machine Learning
Data Analysis, Statistics, Machine Learning Leland Wilkinsn Adjunct Prfessr UIC Cmputer Science Chief Scien
More informationExponential Functions, Growth and Decay
Name..Class. Date. Expnential Functins, Grwth and Decay Essential questin: What are the characteristics f an expnential junctin? In an expnential functin, the variable is an expnent. The parent functin
More informationIn SMV I. IAML: Support Vector Machines II. This Time. The SVM optimization problem. We saw:
In SMV I IAML: Supprt Vectr Machines II Nigel Gddard Schl f Infrmatics Semester 1 We sa: Ma margin trick Gemetry f the margin and h t cmpute it Finding the ma margin hyperplane using a cnstrained ptimizatin
More informationk-nearest Neighbor How to choose k Average of k points more reliable when: Large k: noise in attributes +o o noise in class labels
Mtivating Example Memry-Based Learning Instance-Based Learning K-earest eighbr Inductive Assumptin Similar inputs map t similar utputs If nt true => learning is impssible If true => learning reduces t
More informationTree Structured Classifier
Tree Structured Classifier Reference: Classificatin and Regressin Trees by L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stne, Chapman & Hall, 98. A Medical Eample (CART): Predict high risk patients
More informationFlipping Physics Lecture Notes: Simple Harmonic Motion Introduction via a Horizontal Mass-Spring System
Flipping Physics Lecture Ntes: Simple Harmnic Mtin Intrductin via a Hrizntal Mass-Spring System A Hrizntal Mass-Spring System is where a mass is attached t a spring, riented hrizntally, and then placed
More informationFlipping Physics Lecture Notes: Simple Harmonic Motion Introduction via a Horizontal Mass-Spring System
Flipping Physics Lecture Ntes: Simple Harmnic Mtin Intrductin via a Hrizntal Mass-Spring System A Hrizntal Mass-Spring System is where a mass is attached t a spring, riented hrizntally, and then placed
More informationMODULE FOUR. This module addresses functions. SC Academic Elementary Algebra Standards:
MODULE FOUR This mdule addresses functins SC Academic Standards: EA-3.1 Classify a relatinship as being either a functin r nt a functin when given data as a table, set f rdered pairs, r graph. EA-3.2 Use
More informationWhat is Statistical Learning?
What is Statistical Learning? Sales 5 10 15 20 25 Sales 5 10 15 20 25 Sales 5 10 15 20 25 0 50 100 200 300 TV 0 10 20 30 40 50 Radi 0 20 40 60 80 100 Newspaper Shwn are Sales vs TV, Radi and Newspaper,
More informationEEO 401 Digital Signal Processing Prof. Mark Fowler
EEO 401 Digital Signal Prcessing Prf. Mark Fwler Intrductin Nte Set #1 ading Assignment: Ch. 1 f Prakis & Manlakis 1/13 Mdern systems generally DSP Scenari get a cntinuus-time signal frm a sensr a cnt.-time
More informationWe say that y is a linear function of x if. Chapter 13: The Correlation Coefficient and the Regression Line
Chapter 13: The Crrelatin Cefficient and the Regressin Line We begin with a sme useful facts abut straight lines. Recall the x, y crdinate system, as pictured belw. 3 2 1 y = 2.5 y = 0.5x 3 2 1 1 2 3 1
More informationFive Whys How To Do It Better
Five Whys Definitin. As explained in the previus article, we define rt cause as simply the uncvering f hw the current prblem came int being. Fr a simple causal chain, it is the entire chain. Fr a cmplex
More informationPhysics 212. Lecture 12. Today's Concept: Magnetic Force on moving charges. Physics 212 Lecture 12, Slide 1
Physics 1 Lecture 1 Tday's Cncept: Magnetic Frce n mving charges F qv Physics 1 Lecture 1, Slide 1 Music Wh is the Artist? A) The Meters ) The Neville rthers C) Trmbne Shrty D) Michael Franti E) Radiatrs
More informationCHAPTER 8b Static Equilibrium Units
CHAPTER 8b Static Equilibrium Units The Cnditins fr Equilibrium Slving Statics Prblems Stability and Balance Elasticity; Stress and Strain The Cnditins fr Equilibrium An bject with frces acting n it, but
More informationAP Statistics Notes Unit Two: The Normal Distributions
AP Statistics Ntes Unit Tw: The Nrmal Distributins Syllabus Objectives: 1.5 The student will summarize distributins f data measuring the psitin using quartiles, percentiles, and standardized scres (z-scres).
More informationWho is the Holy Spirit?
ill at w w this h t h in SS est abut erence u O q L G ka iff hink : As m t t es a d K S k A the n ma. wn help rmati ur Jesus. y f t u inf e life ab h iple in t alk a disc f T : RE ce as ece t i A p SH
More informationI. Analytical Potential and Field of a Uniform Rod. V E d. The definition of electric potential difference is
Length L>>a,b,c Phys 232 Lab 4 Ch 17 Electric Ptential Difference Materials: whitebards & pens, cmputers with VPythn, pwer supply & cables, multimeter, crkbard, thumbtacks, individual prbes and jined prbes,
More informationECE 5318/6352 Antenna Engineering. Spring 2006 Dr. Stuart Long. Chapter 6. Part 7 Schelkunoff s Polynomial
ECE 538/635 Antenna Engineering Spring 006 Dr. Stuart Lng Chapter 6 Part 7 Schelkunff s Plynmial 7 Schelkunff s Plynmial Representatin (fr discrete arrays) AF( ψ ) N n 0 A n e jnψ N number f elements in
More informationPhys. 344 Ch 7 Lecture 8 Fri., April. 10 th,
Phys. 344 Ch 7 Lecture 8 Fri., April. 0 th, 009 Fri. 4/0 8. Ising Mdel f Ferrmagnets HW30 66, 74 Mn. 4/3 Review Sat. 4/8 3pm Exam 3 HW Mnday: Review fr est 3. See n-line practice test lecture-prep is t
More informationPLEASURE TEST SERIES (XI) - 07 By O.P. Gupta (For stuffs on Math, click at theopgupta.com)
A Cmpilatin By : OP Gupta (WhatsApp @ +9-9650 50 80) Fr mre stuffs n Maths, please visit : wwwtheopguptacm Time Allwed : 80 Minutes Max Marks : 00 SECTION A Questin numbers 0 t 0 carry mark each x x 5
More informationChapter Summary. Mathematical Induction Strong Induction Recursive Definitions Structural Induction Recursive Algorithms
Chapter 5 1 Chapter Summary Mathematical Inductin Strng Inductin Recursive Definitins Structural Inductin Recursive Algrithms Sectin 5.1 3 Sectin Summary Mathematical Inductin Examples f Prf by Mathematical
More informationIn the OLG model, agents live for two periods. they work and divide their labour income between consumption and
1 The Overlapping Generatins Mdel (OLG) In the OLG mdel, agents live fr tw perids. When ung the wrk and divide their labur incme between cnsumptin and savings. When ld the cnsume their savings. As the
More informationFigure 1a. A planar mechanism.
ME 5 - Machine Design I Fall Semester 0 Name f Student Lab Sectin Number EXAM. OPEN BOOK AND CLOSED NOTES. Mnday, September rd, 0 Write n ne side nly f the paper prvided fr yur slutins. Where necessary,
More informationGroup Analysis: Hands-On
Grup Analysis: Hands-On Gang Chen SSCC/NIMH/NIH/HHS 3/19/16 1 Make sure yu have the files!! Under directry grup_analysis_hands_n/! Slides: GrupAna_HO.pdf! Data: AFNI_data6/GrupAna_cases/! In case yu dn
More informationStat 5100 Handout #26: Variations on OLS Linear Regression (Ch. 11, 13)
Stat 5100 Handout #26: Variations on OLS Linear Regression (Ch. 11, 13) 1. Weighted Least Squares (textbook 11.1) Recall regression model Y = β 0 + β 1 X 1 +... + β p 1 X p 1 + ε in matrix form: (Ch. 5,
More informationMath Foundations 10 Work Plan
Math Fundatins 10 Wrk Plan Units / Tpics 10.1 Demnstrate understanding f factrs f whle numbers by: Prime factrs Greatest Cmmn Factrs (GCF) Least Cmmn Multiple (LCM) Principal square rt Cube rt Time Frame
More informationDepartment of Economics, University of California, Davis Ecn 200C Micro Theory Professor Giacomo Bonanno. Insurance Markets
Department f Ecnmics, University f alifrnia, Davis Ecn 200 Micr Thery Prfessr Giacm Bnann Insurance Markets nsider an individual wh has an initial wealth f. ith sme prbability p he faces a lss f x (0
More informationThermodynamics and Equilibrium
Thermdynamics and Equilibrium Thermdynamics Thermdynamics is the study f the relatinship between heat and ther frms f energy in a chemical r physical prcess. We intrduced the thermdynamic prperty f enthalpy,
More informationSection 5.8 Notes Page Exponential Growth and Decay Models; Newton s Law
Sectin 5.8 Ntes Page 1 5.8 Expnential Grwth and Decay Mdels; Newtn s Law There are many applicatins t expnential functins that we will fcus n in this sectin. First let s lk at the expnential mdel. Expnential
More informationNGSS High School Physics Domain Model
NGSS High Schl Physics Dmain Mdel Mtin and Stability: Frces and Interactins HS-PS2-1: Students will be able t analyze data t supprt the claim that Newtn s secnd law f mtin describes the mathematical relatinship
More informationLCAO APPROXIMATIONS OF ORGANIC Pi MO SYSTEMS The allyl system (cation, anion or radical).
Principles f Organic Chemistry lecture 5, page LCAO APPROIMATIONS OF ORGANIC Pi MO SYSTEMS The allyl system (catin, anin r radical).. Draw mlecule and set up determinant. 2 3 0 3 C C 2 = 0 C 2 3 0 = -
More information1 Course Notes in Introductory Physics Jeffrey Seguritan
Intrductin & Kinematics I Intrductin Quickie Cncepts Units SI is standard system f units used t measure physical quantities. Base units that we use: meter (m) is standard unit f length kilgram (kg) is
More informationPlease Stop Laughing at Me and Pay it Forward Final Writing Assignment
Kirk Please Stp Laughing at Me and Pay it Frward Final Writing Assignment Our fcus fr the past few mnths has been n bullying and hw we treat ther peple. We ve played sme games, read sme articles, read
More informationSections 15.1 to 15.12, 16.1 and 16.2 of the textbook (Robbins-Miller) cover the materials required for this topic.
Tpic : AC Fundamentals, Sinusidal Wavefrm, and Phasrs Sectins 5. t 5., 6. and 6. f the textbk (Rbbins-Miller) cver the materials required fr this tpic.. Wavefrms in electrical systems are current r vltage
More informationTrigonometric Functions. Concept Category 3
Trignmetric Functins Cncept Categry 3 Gals 6 basic trig functins (gemetry) Special triangles Inverse trig functins (t find the angles) Unit Circle: Trig identities a b c 2 2 2 The Six Basic Trig functins
More informationREADING STATECHART DIAGRAMS
READING STATECHART DIAGRAMS Figure 4.48 A Statechart diagram with events The diagram in Figure 4.48 shws all states that the bject plane can be in during the curse f its life. Furthermre, it shws the pssible
More informationKinetics of Particles. Chapter 3
Kinetics f Particles Chapter 3 1 Kinetics f Particles It is the study f the relatins existing between the frces acting n bdy, the mass f the bdy, and the mtin f the bdy. It is the study f the relatin between
More informationLecture 13: Markov Chain Monte Carlo. Gibbs sampling
Lecture 13: Markv hain Mnte arl Gibbs sampling Gibbs sampling Markv chains 1 Recall: Apprximate inference using samples Main idea: we generate samples frm ur Bayes net, then cmpute prbabilities using (weighted)
More informationRelationships Between Frequency, Capacitance, Inductance and Reactance.
P Physics Relatinships between f,, and. Relatinships Between Frequency, apacitance, nductance and Reactance. Purpse: T experimentally verify the relatinships between f, and. The data cllected will lead
More informationHubble s Law PHYS 1301
1 PHYS 1301 Hubble s Law Why: The lab will verify Hubble s law fr the expansin f the universe which is ne f the imprtant cnsequences f general relativity. What: Frm measurements f the angular size and
More information5 th grade Common Core Standards
5 th grade Cmmn Cre Standards In Grade 5, instructinal time shuld fcus n three critical areas: (1) develping fluency with additin and subtractin f fractins, and develping understanding f the multiplicatin
More information