CS 2750 Machine Learning. Lecture 7. Linear regression. CS 2750 Machine Learning. Linear regression. is a linear combination of input components x
|
|
- Pierce Hodge
- 5 years ago
- Views:
Transcription
1 CS 75 Mache Learg Lecture 7 Lear regresso Mlos Hauskrecht los@cs.ptt.edu 59 Seott Square CS 75 Mache Learg Lear regresso Fucto f : X Y s a lear cobato of put copoets f K d d K k - paraeters eghts + d Bas ter f Iput vector d d CS 75 Mache Learg
2 Lear regresso. Error. Data: D < > Fucto: f We ould lke to have f for all.. Error fucto easures ho uch our predctos devate fro the desred asers Mea-squared error.. Learg: We at to fd the eghts zg the error! f CS 75 Mache Learg Lear regresso. Eaple desoal put CS 75 Mache Learg
3 Lear regresso. Eaple. desoal put CS 75 Mache Learg Solvg lear regresso The optal set of eghts satsfes: T Leads to a sste of lear equatos SLE th d+ ukos of the for A b + + K+ + K+ d d Soluto to SLE: atr verso A b CS 75 Mache Learg
4 Gradet descet soluto Goal: the eght optzato the lear regresso odel Error f.. Iteratve soluto: Gradet descet frst order ethod Idea: Adust eghts the drecto that proves the Error The gradet tells us hat s the rght drecto α Error α > - a learg rate scales the gradet chages CS 75 Mache Learg Gradet descet ethod Desced usg the gradet forato Error Error * * Drecto of the descet Chage the value of accordg to the gradet α Error CS 75 Mache Learg
5 Gradet descet ethod Iteratvel approaches the optu of the Error fucto Error CS 75 Mache Learg -th eght: Ole gradet ethod T Lear odel f O-le error ole Error f O-le algorth: geerates a sequece of ole updates -th update step th : D < > Error α + α f Fed learg rate: - Use a sall costat α C Aealed learg rate: α - Graduall rescales chages CS 75 Mache Learg
6 O-le learg. Eaple CS 75 Mache Learg Etesos of sple lear odel Replace puts to lear uts th feature bass fuctos to odel oleartes f + φ φ f φ φ - a arbtrar fucto of d φ The sae techques as before to lear the eghts CS 75 Mache Learg
7 Addtve lear odels Models lear the paraeters e at to ft f + k φ k k... - paraeters φ φ... φ - feature or bass fuctos Bass fuctos eaples: a hgher order poloal oe-desoal put φ φ φ Multdesoal quadratc φ φ φ φ 5 Other tpes of bass fuctos φ s φ cos φ CS 75 Mache Learg Fttg addtve lear odels Error fucto /.. f Assue: φ φ φ f φ.. Leads to a sste of lear equatos K φ φ + K+ φ φ + K+ φ φ φ Ca be solved eactl lke the lear case CS 75 Mache Learg
8 Eaple. Regresso th poloals. Regresso th poloals of degree Data pots: pars of < > Feature fuctos: feature fuctos φ K Fucto to lear: f + φ φ φ + φ CS 75 Mache Learg Learg th feature fuctos. Fucto to lear: f + φ O le gradet update for the <> par + α f k +α f φ Gradet updates are of the sae for as the lear ad logstc regresso odels CS 75 Mache Learg
9 Eaple. Regresso th poloals. Eaple: Regresso th poloals of degree + f φ O le update for <> par + α f + + α f CS 75 Mache Learg Multdesoal addtve odel eaple CS 75 Mache Learg
10 Multdesoal addtve odel eaple CS 75 Mache Learg Statstcal odel of regresso A geeratve odel: f + ε f s a deterstc fucto ε s a rado ose t represets thgs e caot capture th f e.g. ε ~ N CS 75 Mache Learg
11 Statstcal odel of regresso Assue a geeratve odel: f + ε here T f s a lear odel ad ε ~ N The: f E odels the ea of outputs for ad the ose ε odels devatos fro the ea The odel defes the codtoal dest of gve p ep f π CS 75 Mache Learg ML estato of the paraeters lkelhood of predctos the probablt of observg outputs D gve ad s L D p Mau lkelhood estato of paraeters paraeters azg the lkelhood of predctos * arg a Log-lkelhood trck for the ML optzato Mazg the log-lkelhood s equvalet to azg the lkelhood p CS 75 Mache Learg l D log L D log p
12 CS 75 Mache Learg ML estato of the paraeters Usg codtoal dest We ca rerte the log-lkelhood as Mazg th regard to s equvalet to zg squared error fucto p D L D l log log c f p log + C f ] ep[ f p π CS 75 Mache Learg ML estato of paraeters Crtera based o ea squares error fucto ad the log lkelhood of the output are related We ko ho to optze paraeters the sae approach as used for the least squares ft But hat s the ML estate of the varace of the ose? Maze th respect to varace log c p ole + D l f * ˆ ea squared predcto error for the best predctor
13 Regularzed lear regresso If the uber of paraeters s large relatve to the uber of data pots used to tra the odel e face the threat of overft geeralzato error of the odel goes up The predcto accurac ca be ofte proved b settg soe coeffcets to zero Icreases the bas reduces the varace of estates Solutos: Subset selecto Rdge regresso Prcpal copoet regresso Net: rdge regresso CS 75 Mache Learg Rdge regresso Error fucto for the stadard least squares estates: T.. * T We seek: arg Rdge regresso: T + λ Where.. d.. ad λ What does the e error fucto do? CS 75 Mache Learg
14 Rdge regresso Stadard regresso: Rdge regresso: d T.. T +.. pealzes o-zero eghts th the cost proportoal to λ a shrkage coeffcet If a put attrbute has a sall effect o provg the error fucto t s shut do b the pealt ter Icluso of a shrkage pealt s ofte referred to as regularzato CS 75 Mache Learg λ Regularzed lear regresso Ho to solve the least squares proble f the error fucto s erched b the regularzato ter λ? Aser: The soluto to the optal set of eghts s obtaed aga b solvg a set of lear equato. Stadard lear regresso: T Soluto: * X X T X Regularzed lear regresso: T here X s a d atr th ros correspodg to eaples ad colus to puts * λi + X X T X T CS 75 Mache Learg
CS 2750 Machine Learning. Lecture 8. Linear regression. CS 2750 Machine Learning. Linear regression. is a linear combination of input components x
CS 75 Mache Learg Lecture 8 Lear regresso Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Learg Lear regresso Fucto f : X Y s a lear combato of put compoets f + + + K d d K k - parameters
More informationCS 2750 Machine Learning Lecture 8. Linear regression. Supervised learning. a set of n examples
CS 75 Mache Learg Lecture 8 Lear regresso Mlos Hauskrecht los@cs.tt.eu 59 Seott Square Suervse learg Data: D { D D.. D} a set of eales D s a ut vector of sze s the esre outut gve b a teacher Obectve: lear
More informationSupervised learning: Linear regression Logistic regression
CS 57 Itroducto to AI Lecture 4 Supervsed learg: Lear regresso Logstc regresso Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 57 Itro to AI Data: D { D D.. D D Supervsed learg d a set of eamples s
More informationGenerative classification models
CS 75 Mache Learg Lecture Geeratve classfcato models Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Data: D { d, d,.., d} d, Classfcato represets a dscrete class value Goal: lear f : X Y Bar classfcato
More informationLinear regression (cont.) Linear methods for classification
CS 75 Mache Lear Lecture 7 Lear reresso cot. Lear methods for classfcato Mlos Hausrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Lear Coeffcet shrae he least squares estmates ofte have lo bas but hh
More informationLinear regression (cont) Logistic regression
CS 7 Fouatos of Mache Lear Lecture 4 Lear reresso cot Lostc reresso Mlos Hausrecht mlos@cs.ptt.eu 539 Seott Square Lear reresso Vector efto of the moel Iclue bas costat the put vector f - parameters ehts
More informationLinear Regression Linear Regression with Shrinkage. Some slides are due to Tommi Jaakkola, MIT AI Lab
Lear Regresso Lear Regresso th Shrkage Some sldes are due to Tomm Jaakkola, MIT AI Lab Itroducto The goal of regresso s to make quattatve real valued predctos o the bass of a vector of features or attrbutes.
More informationSome Different Perspectives on Linear Least Squares
Soe Dfferet Perspectves o Lear Least Squares A stadard proble statstcs s to easure a respose or depedet varable, y, at fed values of oe or ore depedet varables. Soetes there ests a deterstc odel y f (,,
More informationCS 1675 Introduction to Machine Learning Lecture 12 Support vector machines
CS 675 Itroducto to Mache Learg Lecture Support vector maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Mdterm eam October 9, 7 I-class eam Closed book Stud materal: Lecture otes Correspodg chapters
More informationAlgorithms behind the Correlation Setting Window
Algorths behd the Correlato Settg Wdow Itroducto I ths report detaled forato about the correlato settg pop up wdow s gve. See Fgure. Ths wdow s obtaed b clckg o the rado butto labelled Kow dep the a scree
More informationClassification : Logistic regression. Generative classification model.
CS 75 Mache Lear Lecture 8 Classfcato : Lostc reresso. Geeratve classfcato model. Mlos Hausrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Lear Bar classfcato o classes Y {} Our oal s to lear to classf
More informationSupport vector machines II
CS 75 Mache Learg Lecture Support vector maches II Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Learl separable classes Learl separable classes: here s a hperplae that separates trag staces th o error
More informationOverview. Basic concepts of Bayesian learning. Most probable model given data Coin tosses Linear regression Logistic regression
Overvew Basc cocepts of Bayesa learg Most probable model gve data Co tosses Lear regresso Logstc regresso Bayesa predctos Co tosses Lear regresso 30 Recap: regresso problems Iput to learg problem: trag
More informationBinary classification: Support Vector Machines
CS 57 Itroducto to AI Lecture 6 Bar classfcato: Support Vector Maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 57 Itro to AI Supervsed learg Data: D { D, D,.., D} a set of eamples D, (,,,,,
More informationRegression and the LMS Algorithm
CSE 556: Itroducto to Neural Netorks Regresso ad the LMS Algorthm CSE 556: Regresso 1 Problem statemet CSE 556: Regresso Lear regresso th oe varable Gve a set of N pars of data {, d }, appromate d b a
More informationDimensionality reduction Feature selection
CS 750 Mache Learg Lecture 3 Dmesoalty reducto Feature selecto Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 750 Mache Learg Dmesoalty reducto. Motvato. Classfcato problem eample: We have a put data
More informationObjectives of Multiple Regression
Obectves of Multple Regresso Establsh the lear equato that best predcts values of a depedet varable Y usg more tha oe eplaator varable from a large set of potetal predctors {,,... k }. Fd that subset of
More informationCSE 5526: Introduction to Neural Networks Linear Regression
CSE 556: Itroducto to Neural Netorks Lear Regresso Part II 1 Problem statemet Part II Problem statemet Part II 3 Lear regresso th oe varable Gve a set of N pars of data , appromate d by a lear fucto
More informationThe Mathematics of Portfolio Theory
The Matheatcs of Portfolo Theory The rates of retur of stocks, ad are as follows Market odtos state / scearo) earsh Neutral ullsh Probablty 0. 0.5 0.3 % 5% 9% -3% 3% % 5% % -% Notato: R The retur of stock
More information3D Reconstruction from Image Pairs. Reconstruction from Multiple Views. Computing Scene Point from Two Matching Image Points
D Recostructo fro Iage ars Recostructo fro ultple Ves Dael Deetho Fd terest pots atch terest pots Copute fudaetal atr F Copute caera atrces ad fro F For each atchg age pots ad copute pot scee Coputg Scee
More informationMachine Learning. Introduction to Regression. Le Song. CSE6740/CS7641/ISYE6740, Fall 2012
Mache Learg CSE6740/CS764/ISYE6740, Fall 0 Itroducto to Regresso Le Sog Lecture 4, August 30, 0 Based o sldes from Erc g, CMU Readg: Chap. 3, CB Mache learg for apartmet hutg Suppose ou are to move to
More information7.0 Equality Contraints: Lagrange Multipliers
Systes Optzato 7.0 Equalty Cotrats: Lagrage Multplers Cosder the zato of a o-lear fucto subject to equalty costrats: g f() R ( ) 0 ( ) (7.) where the g ( ) are possbly also olear fuctos, ad < otherwse
More informationBayes (Naïve or not) Classifiers: Generative Approach
Logstc regresso Bayes (Naïve or ot) Classfers: Geeratve Approach What do we mea by Geeratve approach: Lear p(y), p(x y) ad the apply bayes rule to compute p(y x) for makg predctos Ths s essetally makg
More informationCS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 17
CS434a/54a: Patter Recogto Prof. Olga Vesler Lecture 7 Today Paraetrc Usupervsed Learg Expectato Maxato (EM) oe of the ost useful statstcal ethods oldest verso 958 (Hartley) seal paper 977 (Depster et
More informationA New Method for Solving Fuzzy Linear. Programming by Solving Linear Programming
ppled Matheatcal Sceces Vol 008 o 50 7-80 New Method for Solvg Fuzzy Lear Prograg by Solvg Lear Prograg S H Nasser a Departet of Matheatcs Faculty of Basc Sceces Mazadara Uversty Babolsar Ira b The Research
More informationSupport vector machines
CS 75 Mache Learg Lecture Support vector maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Learg Outle Outle: Algorthms for lear decso boudary Support vector maches Mamum marg hyperplae.
More informationSolving optimal margin classifier
Solvg optal arg classfer Recall our opt proble: s s equvalet to Wrte te Lagraga: Recall tat * ca be reforulated as No e solve ts dual proble: b b + s.t a b b + s.t 0 [ ] + b b L * a b b L 0 a b b L 0 ***
More informationSTA 108 Applied Linear Models: Regression Analysis Spring Solution for Homework #1
STA 08 Appled Lear Models: Regresso Aalyss Sprg 0 Soluto for Homework #. Let Y the dollar cost per year, X the umber of vsts per year. The the mathematcal relato betwee X ad Y s: Y 300 + X. Ths s a fuctoal
More informationGeneralized Linear Regression with Regularization
Geeralze Lear Regresso wth Regularzato Zoya Bylsk March 3, 05 BASIC REGRESSION PROBLEM Note: I the followg otes I wll make explct what s a vector a what s a scalar usg vec t or otato, to avo cofuso betwee
More information3.1 Introduction to Multinomial Logit and Probit
ES3008 Ecooetrcs Lecture 3 robt ad Logt - Multoal 3. Itroducto to Multoal Logt ad robt 3. Estato of β 3. Itroducto to Multoal Logt ad robt The ultoal Logt odel s used whe there are several optos (ad therefore
More informationSimple Linear Regression
Statstcal Methods I (EST 75) Page 139 Smple Lear Regresso Smple regresso applcatos are used to ft a model descrbg a lear relatoshp betwee two varables. The aspects of least squares regresso ad correlato
More informationECE 194C Target Classification in Sensor Networks Problem. Fundamental problem in pattern recognition.
ECE 94C arget Classfcato Sesor Netorks.ece.ucsb.edu/Facult/Ilts/ece94c Proble Gve sgature of a target, e.g. sesc, acoustc, vdeo. Detere hch categor the sgature belogs to Fudaetal proble patter recogto.
More informationChapter Two. An Introduction to Regression ( )
ubject: A Itroducto to Regresso Frst tage Chapter Two A Itroducto to Regresso (018-019) 1 pg. ubject: A Itroducto to Regresso Frst tage A Itroducto to Regresso Regresso aalss s a statstcal tool for the
More informationDATA DOMAIN DATA DOMAIN
3//6 Coprght otce: Most ages these sldes are Gozalez ad oods Pretce-Hall Note: ages are [spatall] ostatoar sgals. e eed tools to aalze the locall at dfferet resolutos e ca do ths the data doa or sutable
More informationresidual. (Note that usually in descriptions of regression analysis, upper-case
Regresso Aalyss Regresso aalyss fts or derves a model that descres the varato of a respose (or depedet ) varale as a fucto of oe or more predctor (or depedet ) varales. The geeral regresso model s oe of
More informationAn Introduction to. Support Vector Machine
A Itroducto to Support Vector Mache Support Vector Mache (SVM) A classfer derved from statstcal learg theory by Vapk, et al. 99 SVM became famous whe, usg mages as put, t gave accuracy comparable to eural-etwork
More informationOrdinary Least Squares Regression. Simple Regression. Algebra and Assumptions.
Ordary Least Squares egresso. Smple egresso. Algebra ad Assumptos. I ths part of the course we are gog to study a techque for aalysg the lear relatoshp betwee two varables Y ad X. We have pars of observatos
More information12.2 Estimating Model parameters Assumptions: ox and y are related according to the simple linear regression model
1. Estmatg Model parameters Assumptos: ox ad y are related accordg to the smple lear regresso model (The lear regresso model s the model that says that x ad y are related a lear fasho, but the observed
More informationKURODA S METHOD FOR CONSTRUCTING CONSISTENT INPUT-OUTPUT DATA SETS. Peter J. Wilcoxen. Impact Research Centre, University of Melbourne.
KURODA S METHOD FOR CONSTRUCTING CONSISTENT INPUT-OUTPUT DATA SETS by Peter J. Wlcoxe Ipact Research Cetre, Uversty of Melboure Aprl 1989 Ths paper descrbes a ethod that ca be used to resolve cossteces
More informationConstruction of Composite Indices in Presence of Outliers
Costructo of Coposte dces Presece of Outlers SK Mshra Dept. of Ecoocs North-Easter Hll Uversty Shllog (da). troducto: Oftetes we requre costructg coposte dces by a lear cobato of a uber of dcator varables.
More informationLinear Regression with One Regressor
Lear Regresso wth Oe Regressor AIM QA.7. Expla how regresso aalyss ecoometrcs measures the relatoshp betwee depedet ad depedet varables. A regresso aalyss has the goal of measurg how chages oe varable,
More informationThe theoretical background of
he theoretcal backgroud of -echologes he theoretcal backgroud of FactSage he followg sldes gve a abrdged overvew of the ajor uderlyg prcples of the calculatoal odules of FactSage. -echologes he bbs Eergy
More informationESS Line Fitting
ESS 5 014 17. Le Fttg A very commo problem data aalyss s lookg for relatoshpetwee dfferet parameters ad fttg les or surfaces to data. The smplest example s fttg a straght le ad we wll dscuss that here
More informationCS 2750 Machine Learning Lecture 5. Density estimation. Density estimation
CS 750 Mache Learg Lecture 5 esty estmato Mlos Hausrecht mlos@tt.edu 539 Seott Square esty estmato esty estmato: s a usuervsed learg roblem Goal: Lear a model that rereset the relatos amog attrbutes the
More information( ) = ( ) ( ) Chapter 13 Asymptotic Theory and Stochastic Regressors. Stochastic regressors model
Chapter 3 Asmptotc Theor ad Stochastc Regressors The ature of eplaator varable s assumed to be o-stochastc or fed repeated samples a regresso aalss Such a assumpto s approprate for those epermets whch
More information15-381: Artificial Intelligence. Regression and neural networks (NN)
5-38: Artfcal Itellece Reresso ad eural etorks NN) Mmck the bra I the earl das of AI there as a lot of terest develop models that ca mmc huma thk. Whle o oe ke eactl ho the bra orks ad, eve thouh there
More informationAn Approach to Solve Linear Equations Using Time- Variant Adaptation Based Hybrid Evolutionary Algorithm
A Approach to Solve Lear Equatos Usg Te- Varat Adaptato Based Hbrd Evolutoar Algorth 1 A. R. M. Jalal Udd Jaal, 2 M. M. A. Hashe, ad 1 Md. Bazlar Raha. 1 Departet of Matheatcs Khula Uverst of Egeerg ad
More informationDepartment of Mathematics UNIVERSITY OF OSLO. FORMULAS FOR STK4040 (version 1, September 12th, 2011) A - Vectors and matrices
Deartet of Matheatcs UNIVERSITY OF OSLO FORMULAS FOR STK4040 (verso Seteber th 0) A - Vectors ad atrces A) For a x atrx A ad a x atrx B we have ( AB) BA A) For osgular square atrces A ad B we have ( )
More informationLecture 8: Linear Regression
Lecture 8: Lear egresso May 4, GENOME 56, Sprg Goals Develop basc cocepts of lear regresso from a probablstc framework Estmatg parameters ad hypothess testg wth lear models Lear regresso Su I Lee, CSE
More informationD. L. Bricker, 2002 Dept of Mechanical & Industrial Engineering The University of Iowa. CPL/XD 12/10/2003 page 1
D. L. Brcker, 2002 Dept of Mechacal & Idustral Egeerg The Uversty of Iowa CPL/XD 2/0/2003 page Capactated Plat Locato Proble: Mze FY + C X subject to = = j= where Y = j= X D, j =, j X SY, =,... X 0, =,
More informationMaximum Likelihood Estimation
Marquette Uverst Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Coprght 08 b Marquette Uverst Maxmum Lkelhood Estmato We have bee sag that ~
More informationPGE 310: Formulation and Solution in Geosystems Engineering. Dr. Balhoff. Interpolation
PGE 30: Formulato ad Soluto Geosystems Egeerg Dr. Balhoff Iterpolato Numercal Methods wth MATLAB, Recktewald, Chapter 0 ad Numercal Methods for Egeers, Chapra ad Caale, 5 th Ed., Part Fve, Chapter 8 ad
More informationGlobal Optimization for Solving Linear Non-Quadratic Optimal Control Problems
Joural of Appled Matheatcs ad Physcs 06 4 859-869 http://wwwscrporg/joural/jap ISSN Ole: 37-4379 ISSN Prt: 37-435 Global Optzato for Solvg Lear No-Quadratc Optal Cotrol Probles Jghao Zhu Departet of Appled
More informationRelations to Other Statistical Methods Statistical Data Analysis with Positive Definite Kernels
Relatos to Other Statstcal Methods Statstcal Data Aalyss wth Postve Defte Kerels Kej Fukuzu Isttute of Statstcal Matheatcs, ROIS Departet of Statstcal Scece, Graduate Uversty for Advaced Studes October
More informationMidterm Exam 1, section 2 (Solution) Thursday, February hour, 15 minutes
coometrcs, CON Sa Fracsco State Uverst Mchael Bar Sprg 5 Mdterm xam, secto Soluto Thursda, Februar 6 hour, 5 mutes Name: Istructos. Ths s closed book, closed otes exam.. No calculators of a kd are allowed..
More informationGene Expression Data Classification with Kernel independent Component Analysis
Research Joural of Matheatcal ad Statstcal Sceces ISSN 2320 6047 Gee Expresso Data Classfcato wth Kerel depedet Copoet Aalyss Abstract Abdallah Bashr Musa College of Matheatcs ad Coputer Scece, Hebe Uversty,
More informationSome results and conjectures about recurrence relations for certain sequences of binomial sums.
Soe results ad coectures about recurrece relatos for certa sequeces of boal sus Joha Cgler Faultät für Matheat Uverstät We A-9 We Nordbergstraße 5 Joha Cgler@uveacat Abstract I a prevous paper [] I have
More information2/20/2013. Topics. Power Flow Part 1 Text: Power Transmission. Power Transmission. Power Transmission. Power Transmission
/0/0 Topcs Power Flow Part Text: 0-0. Power Trassso Revsted Power Flow Equatos Power Flow Proble Stateet ECEGR 45 Power Systes Power Trassso Power Trassso Recall that for a short trassso le, the power
More informationComparison between MSE and MEE Based Component Extraction Approaches to Process Monitoring and Fault Diagnosis
Coparso betwee MSE ad MEE ased Copoet Etracto pproaches to Process Motorg ad Fault Dagoss Zhehua Guo, ad Hog Wag $ School of Mechacal Scece ad Egeerg, Huazhog Uversty of Scece ad echology, Wuha, Hube,
More informationDimensionality reduction Feature selection
CS 675 Itroucto to ache Learg Lecture Dmesoalty reucto Feature selecto los Hauskrecht mlos@cs.ptt.eu 539 Seott Square Dmesoalty reucto. otvato. L methos are sestve to the mesoalty of ata Questo: Is there
More informationLecture Notes Types of economic variables
Lecture Notes 3 1. Types of ecoomc varables () Cotuous varable takes o a cotuum the sample space, such as all pots o a le or all real umbers Example: GDP, Polluto cocetrato, etc. () Dscrete varables fte
More informationLine Fitting and Regression
Marquette Uverst MSCS6 Le Fttg ad Regresso Dael B. Rowe, Ph.D. Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Coprght 8 b Marquette Uverst Least Squares Regresso MSCS6 For LSR we have pots
More informationA Conventional Approach for the Solution of the Fifth Order Boundary Value Problems Using Sixth Degree Spline Functions
Appled Matheatcs, 1, 4, 8-88 http://d.do.org/1.4/a.1.448 Publshed Ole Aprl 1 (http://www.scrp.org/joural/a) A Covetoal Approach for the Soluto of the Ffth Order Boudary Value Probles Usg Sth Degree Sple
More informationAnalyzing Two-Dimensional Data. Analyzing Two-Dimensional Data
/7/06 Aalzg Two-Dmesoal Data The most commo aaltcal measuremets volve the determato of a ukow cocetrato based o the respose of a aaltcal procedure (usuall strumetal). Such a measuremet requres calbrato,
More informationCH E 374 Computational Methods in Engineering Fall 2007
CH E 7 Computatoal Methods Egeerg Fall 007 Sample Soluto 5. The data o the varato of the rato of stagato pressure to statc pressure (r ) wth Mach umber ( M ) for the flow through a duct are as follows:
More informationSMOOTH SUPPORT VECTOR REGRESSION BASED ON MODIFICATION SPLINE INTERPOLATION
Joural of heoretcal ad Appled Iforato echology 5 th October. Vol. 44 No. 5 - JAI & LLS. All rghts reserved. ISSN: 99-8645 www.att.org E-ISSN: 87-395 OOH SUPPOR VECOR REGRESSION BASED ON ODIFICAION SPLINE
More informationLogistic Regression Classification for Uncertain Data
Abstract Research Joural of Matheatcal ad Statstcal Sceces ISSN 2320 6047 Vol. 2(2), -6, February (204) Res. J. Matheatcal ad Statstcal Sc. Logstc Regresso Classfcato for Ucerta Data Abdallah Bashr Musa
More informationRademacher Complexity. Examples
Algorthmc Foudatos of Learg Lecture 3 Rademacher Complexty. Examples Lecturer: Patrck Rebesch Verso: October 16th 018 3.1 Itroducto I the last lecture we troduced the oto of Rademacher complexty ad showed
More informationBayesian Classification. CS690L Data Mining: Classification(2) Bayesian Theorem: Basics. Bayesian Theorem. Training dataset. Naïve Bayes Classifier
Baa Classfcato CS6L Data Mg: Classfcato() Referece: J. Ha ad M. Kamber, Data Mg: Cocepts ad Techques robablstc learg: Calculate explct probabltes for hypothess, amog the most practcal approaches to certa
More informationKernel-based Methods and Support Vector Machines
Kerel-based Methods ad Support Vector Maches Larr Holder CptS 570 Mache Learg School of Electrcal Egeerg ad Computer Scece Washgto State Uverst Refereces Muller et al. A Itroducto to Kerel-Based Learg
More informationDimensionality Reduction and Learning
CMSC 35900 (Sprg 009) Large Scale Learg Lecture: 3 Dmesoalty Reducto ad Learg Istructors: Sham Kakade ad Greg Shakharovch L Supervsed Methods ad Dmesoalty Reducto The theme of these two lectures s that
More informationChapter Business Statistics: A First Course Fifth Edition. Learning Objectives. Correlation vs. Regression. In this chapter, you learn:
Chapter 3 3- Busess Statstcs: A Frst Course Ffth Edto Chapter 2 Correlato ad Smple Lear Regresso Busess Statstcs: A Frst Course, 5e 29 Pretce-Hall, Ic. Chap 2- Learg Objectves I ths chapter, you lear:
More informationLinear models for classification
CS 75 Mache Lear Lecture 9 Lear modes for cassfcato Mos Hausrecht mos@cs.ptt.edu 539 Seott Square ata: { d d.. d} d Cassfcato represets a dscrete cass vaue Goa: ear f : X Y Bar cassfcato A speca case he
More informationLecture 12 APPROXIMATION OF FIRST ORDER DERIVATIVES
FDM: Appromato of Frst Order Dervatves Lecture APPROXIMATION OF FIRST ORDER DERIVATIVES. INTRODUCTION Covectve term coservato equatos volve frst order dervatves. The smplest possble approach for dscretzato
More informationECON 482 / WH Hong The Simple Regression Model 1. Definition of the Simple Regression Model
ECON 48 / WH Hog The Smple Regresso Model. Defto of the Smple Regresso Model Smple Regresso Model Expla varable y terms of varable x y = β + β x+ u y : depedet varable, explaed varable, respose varable,
More informationLINEAR REGRESSION ANALYSIS
LINEAR REGRESSION ANALYSIS MODULE V Lecture - Correctg Model Iadequaces Through Trasformato ad Weghtg Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur Aalytcal methods for
More informationSimple Linear Regression and Correlation.
Smple Lear Regresso ad Correlato. Correspods to Chapter 0 Tamhae ad Dulop Sldes prepared b Elzabeth Newto (MIT) wth some sldes b Jacquele Telford (Johs Hopks Uverst) Smple lear regresso aalss estmates
More informationå 1 13 Practice Final Examination Solutions - = CS109 Dec 5, 2018
Chrs Pech Fal Practce CS09 Dec 5, 08 Practce Fal Examato Solutos. Aswer: 4/5 8/7. There are multle ways to obta ths aswer; here are two: The frst commo method s to sum over all ossbltes for the rak of
More informationMultiple Regression. More than 2 variables! Grade on Final. Multiple Regression 11/21/2012. Exam 2 Grades. Exam 2 Re-grades
STAT 101 Dr. Kar Lock Morga 11/20/12 Exam 2 Grades Multple Regresso SECTIONS 9.2, 10.1, 10.2 Multple explaatory varables (10.1) Parttog varablty R 2, ANOVA (9.2) Codtos resdual plot (10.2) Trasformatos
More informationRadial Basis Function Networks
Radal Bass Fucto Netorks Radal Bass Fucto Netorks A specal types of ANN that have three layers Iput layer Hdde layer Output layer Mappg from put to hdde layer s olear Mappg from hdde to output layer s
More informationMultivariate Transformation of Variables and Maximum Likelihood Estimation
Marquette Uversty Multvarate Trasformato of Varables ad Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Assocate Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 03 by Marquette Uversty
More informationSolutions to problem set ); (, ) (
Solutos to proble set.. L = ( yp p ); L = ( p p ); y y L, L = yp p, p p = yp p, + p [, p ] y y y = yp + p = L y Here we use for eaple that yp, p = yp p p yp = yp, p = yp : factors that coute ca be treated
More informationExample: Multiple linear regression. Least squares regression. Repetition: Simple linear regression. Tron Anders Moger
Example: Multple lear regresso 5000,00 4000,00 Tro Aders Moger 0.0.007 brthweght 3000,00 000,00 000,00 0,00 50,00 00,00 50,00 00,00 50,00 weght pouds Repetto: Smple lear regresso We defe a model Y = β0
More informationCorrelation and Regression Analysis
Chapter V Correlato ad Regresso Aalss R. 5.. So far we have cosdered ol uvarate dstrbutos. Ma a tme, however, we come across problems whch volve two or more varables. Ths wll be the subject matter of the
More informationSTATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. x, where. = y - ˆ " 1
STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS Recall Assumpto E(Y x) η 0 + η x (lear codtoal mea fucto) Data (x, y ), (x 2, y 2 ),, (x, y ) Least squares estmator ˆ E (Y x) ˆ " 0 + ˆ " x, where ˆ
More informationSTA302/1001-Fall 2008 Midterm Test October 21, 2008
STA3/-Fall 8 Mdterm Test October, 8 Last Name: Frst Name: Studet Number: Erolled (Crcle oe) STA3 STA INSTRUCTIONS Tme allowed: hour 45 mutes Ads allowed: A o-programmable calculator A table of values from
More informationLinear Model Analysis of Observational Data in the Sense of Least Squares Criterion
Aerca Joural of Matheatcal Aalyss, 05, Vol. 3, No., 47-53 Avalable ole at http://pubs.scepub.co/aja/3//5 Scece ad Educato Publshg DOI:0.69/aja-3--5 Lear Model Aalyss of Observatoal Data the Sese of Least
More informationChapter 14 Logistic Regression Models
Chapter 4 Logstc Regresso Models I the lear regresso model X β + ε, there are two types of varables explaatory varables X, X,, X k ad study varable y These varables ca be measured o a cotuous scale as
More informationL5 Polynomial / Spline Curves
L5 Polyomal / Sple Curves Cotets Coc sectos Polyomal Curves Hermte Curves Bezer Curves B-Sples No-Uform Ratoal B-Sples (NURBS) Mapulato ad Represetato of Curves Types of Curve Equatos Implct: Descrbe a
More informationare positive, and the pair A, B is controllable. The uncertainty in is introduced to model control failures.
Lectue 4 8. MRAC Desg fo Affe--Cotol MIMO Systes I ths secto, we cosde MRAC desg fo a class of ult-ut-ult-outut (MIMO) olea systes, whose lat dyacs ae lealy aaetezed, the ucetates satsfy the so-called
More informationENGI 4421 Joint Probability Distributions Page Joint Probability Distributions [Navidi sections 2.5 and 2.6; Devore sections
ENGI 441 Jot Probablty Dstrbutos Page 7-01 Jot Probablty Dstrbutos [Navd sectos.5 ad.6; Devore sectos 5.1-5.] The jot probablty mass fucto of two dscrete radom quattes, s, P ad p x y x y The margal probablty
More informationTESTS BASED ON MAXIMUM LIKELIHOOD
ESE 5 Toy E. Smth. The Basc Example. TESTS BASED ON MAXIMUM LIKELIHOOD To llustrate the propertes of maxmum lkelhood estmates ad tests, we cosder the smplest possble case of estmatg the mea of the ormal
More informationUnsupervised Learning and Other Neural Networks
CSE 53 Soft Computg NOT PART OF THE FINAL Usupervsed Learg ad Other Neural Networs Itroducto Mture Destes ad Idetfablty ML Estmates Applcato to Normal Mtures Other Neural Networs Itroducto Prevously, all
More informationModule 7. Lecture 7: Statistical parameter estimation
Lecture 7: Statstcal parameter estmato Parameter Estmato Methods of Parameter Estmato 1) Method of Matchg Pots ) Method of Momets 3) Mamum Lkelhood method Populato Parameter Sample Parameter Ubased estmato
More informationEstimation of R= P [Y < X] for Two-parameter Burr Type XII Distribution
World Acade of Scece, Egeerg ad Techolog Iteratoal Joural of Matheatcal ad Coputatoal Sceces Vol:4, No:, Estato of R P [Y < X] for Two-paraeter Burr Tpe XII Dstruto H.Paah, S.Asad Iteratoal Scece Ide,
More information4. Standard Regression Model and Spatial Dependence Tests
4. Stadard Regresso Model ad Spatal Depedece Tests Stadard regresso aalss fals the presece of spatal effects. I case of spatal depedeces ad/or spatal heterogeet a stadard regresso model wll be msspecfed.
More informationClass 13,14 June 17, 19, 2015
Class 3,4 Jue 7, 9, 05 Pla for Class3,4:. Samplg dstrbuto of sample mea. The Cetral Lmt Theorem (CLT). Cofdece terval for ukow mea.. Samplg Dstrbuto for Sample mea. Methods used are based o CLT ( Cetral
More informationPart I: Background on the Binomial Distribution
Part I: Bacgroud o the Bomal Dstrbuto A radom varable s sad to have a Beroull dstrbuto f t taes o the value wth probablt "p" ad the value wth probablt " - p". The umber of "successes" "" depedet Beroull
More informationSpreadsheet Problem Solving
1550 1500 CO Emmssos for the US, 1989 000 Class meetg #6 Moday, Sept 14 th CO Emssos (MMT Carbo) y = 1.3x 41090.17 1450 1400 1350 1300 1989 1990 1991 199 1993 1994 1995 1996 1997 1998 1999 000 Year GEEN
More informationThe equation is sometimes presented in form Y = a + b x. This is reasonable, but it s not the notation we use.
INTRODUCTORY NOTE ON LINEAR REGREION We have data of the form (x y ) (x y ) (x y ) These wll most ofte be preseted to us as two colum of a spreadsheet As the topc develops we wll see both upper case ad
More information