CS 2750 Machine Learning Lecture 8. Linear regression. Supervised learning. a set of n examples
|
|
|
- Kelly Ramsey
- 6 years ago
- Views:
Transcription
1 CS 75 Mache Learg Lecture 8 Lear regresso Mlos Hauskrecht los@cs.tt.eu 59 Seott Square Suervse learg Data: D { D D.. D} a set of eales D s a ut vector of sze s the esre outut gve b a teacher Obectve: lear the ag f : X Y s.t. f for all.. Regresso: Y s cotuous Eale: eargs rouct orers coa stock rce Classfcato: Y s screte Eale: hartte gt bar for gt label
2 Suervse learg eales Regresso: Y s cotuous Debt/equt Eargs Future rouct orers Stock rce Data: Debt/equt Eargs Future ro orers Stock rce Lear regresso Fucto f : X Y Y s a lear cobato of ut cooets f k - araeters eghts Bas ter f Iut vector
3 Lear regresso Shorter vector efto of the oel Iclue bas costat the ut vector f k - araeters eghts Iut vector f Lear regresso. Error. Data: D Fucto: f We oul lke to have f for all.. Error fucto easures ho uch our rectos evate fro the esre asers Mea-square error.. Learg: We at to f the eghts zg the error! f
4 Lear regresso. Eale esoal ut Lear regresso. Eale. esoal ut
5 5 Lear regresso. Otzato. We at the eghts zg the error For the otal set of araeters ervatves of the error th resect to each araeter ust be Vector of ervatves:.... f gra Lear regresso. Otzato. efes a set of equatos gra
6 6 Solvg lear regresso B rearragg the ters e get a sste of lear equatos th + ukos A b Solvg lear regresso he otal set of eghts satsfes: Leas to a sste of lear equatos SLE th + ukos of the for Soluto to SLE: Assug X s a ata atr th ros corresog to eales a colus to uts a s vector of oututs the A b b A X X X
7 Graet escet soluto Goal: the eght otzato the lear regresso oel Error f.. A alteratve to SLE soluto: Graet escet Iea: Aust eghts the recto that roves the Error he graet tells us hat s the rght recto Error - a learg rate scales the graet chages Graet escet etho Desce usg the graet forato Error Error * * Drecto of the escet Chage the value of accorg to the graet Error a learg rate scales the graet chages 7
8 Graet escet etho Iteratvel aroaches the otu of the Error fucto Error Batch vs Ole regresso algorth he error fucto efe o the colete ataset D Error f.. We sa e are learg the oel the batch oe: All eales are avalable at the te of learg Weghts are otzes th resect to all trag eales A alteratve s to lear the oel the ole oe Eales are arrvg sequetall Moel eghts are uate after ever eale If eee eales see ca be forgotte - 8
9 Ole graet algorth he error fucto s efe for the colete ataset D Error f Error for oe eale.. ole Error f Ole graet etho: chages eghts after ever eale Error vector for: D Error - Learg rate that ees o the uber of uates Ole graet etho Lear oel f O-le error ole Error f O-le algorth: geerates a sequece of ole uates -th uate ste th : -th eght: D Error f Fe learg rate: - Use a sall costat C Aeale learg rate: - Grauall rescales chages 9
10 Ole regresso algorth Ole-lear-regresso stog_crtero Italze eghts talze =; hle stog_crtero = FALSE select the et ata ot D set learg rate uate eght vector f e retur eghts Avatages: ver eas to leet cotuous ata streas O-le learg. Eale
11 Etesos of sle lear oel Relace uts to lear uts th feature bass fuctos to oel oleartes f - a arbtrar fucto of f Orgal ut Ne ut Lear oel Etesos of sle lear oel Moels lear the araeters e at to ft f k k k... - araeters... - feature or bass fuctos Bass fuctos eales: a hgher orer oloal oe-esoal ut
12 Etesos of sle lear oel Moels lear the araeters e at to ft f k k k Bass fuctos eales: a hgher orer oloal oe-esoal ut Multesoal quaratc... - araeters... - feature or bass fuctos 4 5 Etesos of sle lear oel Moels lear the araeters e at to ft f k k k Bass fuctos eales: a hgher orer oloal oe-esoal ut Multesoal quaratc 4 Other tes of bass fuctos s cos... - araeters... - feature or bass fuctos 5
13 Etesos of sle lear oel Error fucto.. / f φ.. f Leas to a sste of lear equatos φ Assue: Ca be solve eactl lke the lear case he sae techques as for the lear oel to lear the eghts Eale. Regresso th oloals. Regresso th oloals of egree Data staces: ars of Feature fuctos: feature fuctos Fucto to lear: f
14 Fucto to lear: Learg th feature fuctos f O le graet uate for the <> ar f k f Graet uates are of the sae for as the lear regresso oels Lear oel eale
15 No-lear oel Lear regresso oel Lear oel: f Notce: the above oel oes ot tr to ela varato observe s for the ata 5
16 Statstcal oel of regresso A statstcal oel of lear regresso: ~ N s a rao ose reresets thgs e caot cature th 5 E 5 5 Gaussa ose ε ~ N Statstcal oel of regresso A statstcal oel of lear regresso: he cotoal strbuto of gve e E ~ N ~ N 6
17 7 ML estato of the araeters lkelhoo of rectos = the robablt of observg oututs D gve Mau lkelhoo estato of araeters araeters azg the lkelhoo of rectos Log-lkelhoo trck for the ML otzato Mazg the log-lkelhoo s equvalet to azg the lkelhoo D L * a arg D L D l log log ML estato of the araeters Usg cotoal est We ca rerte the log-lkelhoo as D L D l log log c log C ] e[ f D e see a slar eresso before?
18 8 ML estato of the araeters Usg cotoal est We ca rerte the log-lkelhoo as Mazg the rectve log lkelhoo th regar to s equvalet to zg the ea square error fucto D L D l log log c log C ] e[ f ML estato of araeters Crtera base o the ea squares error fucto a the log lkelhoo of the outut are relate We ko ho to otze araeters the sae aroach as use for the least squares ft But hat s the ML estate of the varace of the ose? Maze th resect to varace log c ole D l f * ˆ = ea square recto error for the best rector
CS 2750 Machine Learning. Lecture 7. Linear regression. CS 2750 Machine Learning. Linear regression. is a linear combination of input components x
CS 75 Mache Learg Lecture 7 Lear regresso Mlos Hauskrecht los@cs.ptt.edu 59 Seott Square CS 75 Mache Learg Lear regresso Fucto f : X Y s a lear cobato of put copoets f + + + K d d K k - paraeters eghts
Supervised learning: Linear regression Logistic regression
CS 57 Itroducto to AI Lecture 4 Supervsed learg: Lear regresso Logstc regresso Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 57 Itro to AI Data: D { D D.. D D Supervsed learg d a set of eamples s
CS 2750 Machine Learning. Lecture 8. Linear regression. CS 2750 Machine Learning. Linear regression. is a linear combination of input components x
CS 75 Mache Learg Lecture 8 Lear regresso Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Learg Lear regresso Fucto f : X Y s a lear combato of put compoets f + + + K d d K k - parameters
Linear regression (cont) Logistic regression
CS 7 Fouatos of Mache Lear Lecture 4 Lear reresso cot Lostc reresso Mlos Hausrecht mlos@cs.ptt.eu 539 Seott Square Lear reresso Vector efto of the moel Iclue bas costat the put vector f - parameters ehts
Binary classification: Support Vector Machines
CS 57 Itroducto to AI Lecture 6 Bar classfcato: Support Vector Maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 57 Itro to AI Supervsed learg Data: D { D, D,.., D} a set of eamples D, (,,,,,
Density estimation III. Linear regression.
Lecure 6 Mlos Hauskrec mlos@cs.p.eu 539 Seo Square Des esmao III. Lear regresso. Daa: Des esmao D { D D.. D} D a vecor of arbue values Obecve: r o esmae e uerlg rue probabl srbuo over varables X px usg
Support vector machines II
CS 75 Mache Learg Lecture Support vector maches II Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Learl separable classes Learl separable classes: here s a hperplae that separates trag staces th o error
CS 3710 Advanced Topics in AI Lecture 17. Density estimation. CS 3710 Probabilistic graphical models. Administration
CS 37 Avace Topcs AI Lecture 7 esty estmato Mlos Hauskrecht mlos@cs.ptt.eu 539 Seott Square CS 37 robablstc graphcal moels Amstrato Mterm: A take-home exam week ue o Weesay ovember 5 before the class epes
Generative classification models
CS 75 Mache Learg Lecture Geeratve classfcato models Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Data: D { d, d,.., d} d, Classfcato represets a dscrete class value Goal: lear f : X Y Bar classfcato
CS 1675 Introduction to Machine Learning Lecture 12 Support vector machines
CS 675 Itroducto to Mache Learg Lecture Support vector maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Mdterm eam October 9, 7 I-class eam Closed book Stud materal: Lecture otes Correspodg chapters
CS 2750 Machine Learning Lecture 5. Density estimation. Density estimation
CS 750 Mache Learg Lecture 5 esty estmato Mlos Hausrecht mlos@tt.edu 539 Seott Square esty estmato esty estmato: s a usuervsed learg roblem Goal: Lear a model that rereset the relatos amog attrbutes the
Support vector machines
CS 75 Mache Learg Lecture Support vector maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Learg Outle Outle: Algorthms for lear decso boudary Support vector maches Mamum marg hyperplae.
An Implementation of Integer Programming Techniques in Clustering Algorithm
S. Shebaga Ezhl et al / Ia Joural of oputer Scece a Egeerg (IJSE) A Ipleetato of Iteger rograg echques lusterg Algorth S. Shebaga Ezhl a Dr.. Vayalaksh 2 Departet of Matheatcs Sathyabaa Uversty, hea 9
Linear regression (cont.) Linear methods for classification
CS 75 Mache Lear Lecture 7 Lear reresso cot. Lear methods for classfcato Mlos Hausrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Lear Coeffcet shrae he least squares estmates ofte have lo bas but hh
Some Different Perspectives on Linear Least Squares
Soe Dfferet Perspectves o Lear Least Squares A stadard proble statstcs s to easure a respose or depedet varable, y, at fed values of oe or ore depedet varables. Soetes there ests a deterstc odel y f (,,
M2S1 - EXERCISES 8: SOLUTIONS
MS - EXERCISES 8: SOLUTIONS. As X,..., X P ossoλ, a gve that T ˉX, the usg elemetary propertes of expectatos, we have E ft [T E fx [X λ λ, so that T s a ubase estmator of λ. T X X X Furthermore X X X From
Classification : Logistic regression. Generative classification model.
CS 75 Mache Lear Lecture 8 Classfcato : Lostc reresso. Geeratve classfcato model. Mlos Hausrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Lear Bar classfcato o classes Y {} Our oal s to lear to classf
( x) min. Nonlinear optimization problem without constraints NPP: then. Global minimum of the function f(x)
Objectve fucto f() : he optzato proble cossts of fg a vector of ecso varables belogg to the feasble set of solutos R such that It s eote as: Nolear optzato proble wthout costrats NPP: R f ( ) : R R f f
Generalized Linear Regression with Regularization
Geeralze Lear Regresso wth Regularzato Zoya Bylsk March 3, 05 BASIC REGRESSION PROBLEM Note: I the followg otes I wll make explct what s a vector a what s a scalar usg vec t or otato, to avo cofuso betwee
Algorithms behind the Correlation Setting Window
Algorths behd the Correlato Settg Wdow Itroducto I ths report detaled forato about the correlato settg pop up wdow s gve. See Fgure. Ths wdow s obtaed b clckg o the rado butto labelled Kow dep the a scree
Midterm Exam 1, section 2 (Solution) Thursday, February hour, 15 minutes
coometrcs, CON Sa Fracsco State Uverst Mchael Bar Sprg 5 Mdterm xam, secto Soluto Thursda, Februar 6 hour, 5 mutes Name: Istructos. Ths s closed book, closed otes exam.. No calculators of a kd are allowed..
Solutions to Odd-Numbered End-of-Chapter Exercises: Chapter 17
Itroucto to Ecoometrcs (3 r Upate Eto) by James H. Stock a Mark W. Watso Solutos to O-Numbere E-of-Chapter Exercses: Chapter 7 (Ths erso August 7, 04) 05 Pearso Eucato, Ic. Stock/Watso - Itroucto to Ecoometrcs
ELEMENTS OF NUMBER THEORY. In the following we will use mainly integers and positive integers. - the set of integers - the set of positive integers
ELEMENTS OF NUMBER THEORY I the followg we wll use aly tegers a ostve tegers Ζ = { ± ± ± K} - the set of tegers Ν = { K} - the set of ostve tegers Oeratos o tegers: Ato Each two tegers (ostve tegers) ay
Regression and the LMS Algorithm
CSE 556: Itroducto to Neural Netorks Regresso ad the LMS Algorthm CSE 556: Regresso 1 Problem statemet CSE 556: Regresso Lear regresso th oe varable Gve a set of N pars of data {, d }, appromate d b a
are positive, and the pair A, B is controllable. The uncertainty in is introduced to model control failures.
Lectue 4 8. MRAC Desg fo Affe--Cotol MIMO Systes I ths secto, we cosde MRAC desg fo a class of ult-ut-ult-outut (MIMO) olea systes, whose lat dyacs ae lealy aaetezed, the ucetates satsfy the so-called
An Introduction to. Support Vector Machine
A Itroducto to Support Vector Mache Support Vector Mache (SVM) A classfer derved from statstcal learg theory by Vapk, et al. 99 SVM became famous whe, usg mages as put, t gave accuracy comparable to eural-etwork
The Mathematics of Portfolio Theory
The Matheatcs of Portfolo Theory The rates of retur of stocks, ad are as follows Market odtos state / scearo) earsh Neutral ullsh Probablty 0. 0.5 0.3 % 5% 9% -3% 3% % 5% % -% Notato: R The retur of stock
for each of its columns. A quick calculation will verify that: thus m < dim(v). Then a basis of V with respect to which T has the form: A
Desty of dagoalzable square atrces Studet: Dael Cervoe; Metor: Saravaa Thyagaraa Uversty of Chcago VIGRE REU, Suer 7. For ths etre aer, we wll refer to V as a vector sace over ad L(V) as the set of lear
Theory study about quarter-wave-stack dielectric mirrors
Theor tud about quarter-wave-tack delectrc rror Stratfed edu tratted reflected reflected Stratfed edu tratted cdet cdet T T Frt, coder a wave roagato a tratfed edu. A we kow, a arbtrarl olared lae wave
LINEAR REGRESSION ANALYSIS
LINEAR REGRESSION ANALYSIS MODULE V Lecture - Correctg Model Iadequaces Through Trasformato ad Weghtg Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur Aalytcal methods for
Overview. Basic concepts of Bayesian learning. Most probable model given data Coin tosses Linear regression Logistic regression
Overvew Basc cocepts of Bayesa learg Most probable model gve data Co tosses Lear regresso Logstc regresso Bayesa predctos Co tosses Lear regresso 30 Recap: regresso problems Iput to learg problem: trag
STAT 400 Homework 09 Spring 2018 Dalpiaz UIUC Due: Friday, April 6, 2:00 PM
STAT Homework 9 Sprg 28 Dalpaz UIUC Due: Fray, Aprl 6, 2: PM Exercse f(x, θ) = θ e x/θ, x >, θ > Note that, the momets of ths strbuto are gve by E[X k ] = Ths wll be a useful fact for Exercses 2 a 3. x
CSE 5526: Introduction to Neural Networks Linear Regression
CSE 556: Itroducto to Neural Netorks Lear Regresso Part II 1 Problem statemet Part II Problem statemet Part II 3 Lear regresso th oe varable Gve a set of N pars of data , appromate d by a lear fucto
Dimensionality reduction Feature selection
CS 675 Itroucto to ache Learg Lecture Dmesoalty reucto Feature selecto los Hauskrecht mlos@cs.ptt.eu 539 Seott Square Dmesoalty reucto. otvato. L methos are sestve to the mesoalty of ata Questo: Is there
5. Data Compression. Review of Last Lecture. Outline of the Lecture. Course Overview. Basics of Information Theory: Markku Juntti
: Markku Jutt Overvew The deas of lossless data copresso ad source codg are troduced ad copresso lts are derved. Source The ateral s aly based o Sectos 5. 5.5 of the course book []. Teleco. Laboratory
Lecture Notes Types of economic variables
Lecture Notes 3 1. Types of ecoomc varables () Cotuous varable takes o a cotuum the sample space, such as all pots o a le or all real umbers Example: GDP, Polluto cocetrato, etc. () Dscrete varables fte
3D Reconstruction from Image Pairs. Reconstruction from Multiple Views. Computing Scene Point from Two Matching Image Points
D Recostructo fro Iage ars Recostructo fro ultple Ves Dael Deetho Fd terest pots atch terest pots Copute fudaetal atr F Copute caera atrces ad fro F For each atchg age pots ad copute pot scee Coputg Scee
DATA DOMAIN DATA DOMAIN
3//6 Coprght otce: Most ages these sldes are Gozalez ad oods Pretce-Hall Note: ages are [spatall] ostatoar sgals. e eed tools to aalze the locall at dfferet resolutos e ca do ths the data doa or sutable
2SLS Estimates ECON In this case, begin with the assumption that E[ i
SLS Estmates ECON 3033 Bll Evas Fall 05 Two-Stage Least Squares (SLS Cosder a stadard lear bvarate regresso model y 0 x. I ths case, beg wth the assumto that E[ x] 0 whch meas that OLS estmates of wll
UNIT 7 RANK CORRELATION
UNIT 7 RANK CORRELATION Rak Correlato Structure 7. Itroucto Objectves 7. Cocept of Rak Correlato 7.3 Dervato of Rak Correlato Coeffcet Formula 7.4 Te or Repeate Raks 7.5 Cocurret Devato 7.6 Summar 7.7
Linear Regression Linear Regression with Shrinkage. Some slides are due to Tommi Jaakkola, MIT AI Lab
Lear Regresso Lear Regresso th Shrkage Some sldes are due to Tomm Jaakkola, MIT AI Lab Itroducto The goal of regresso s to make quattatve real valued predctos o the bass of a vector of features or attrbutes.
7.0 Equality Contraints: Lagrange Multipliers
Systes Optzato 7.0 Equalty Cotrats: Lagrage Multplers Cosder the zato of a o-lear fucto subject to equalty costrats: g f() R ( ) 0 ( ) (7.) where the g ( ) are possbly also olear fuctos, ad < otherwse
å 1 13 Practice Final Examination Solutions - = CS109 Dec 5, 2018
Chrs Pech Fal Practce CS09 Dec 5, 08 Practce Fal Examato Solutos. Aswer: 4/5 8/7. There are multle ways to obta ths aswer; here are two: The frst commo method s to sum over all ossbltes for the rak of
Bayes (Naïve or not) Classifiers: Generative Approach
Logstc regresso Bayes (Naïve or ot) Classfers: Geeratve Approach What do we mea by Geeratve approach: Lear p(y), p(x y) ad the apply bayes rule to compute p(y x) for makg predctos Ths s essetally makg
Dimensionality reduction Feature selection
CS 750 Mache Learg Lecture 3 Dmesoalty reducto Feature selecto Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 750 Mache Learg Dmesoalty reducto. Motvato. Classfcato problem eample: We have a put data
ECON 482 / WH Hong The Simple Regression Model 1. Definition of the Simple Regression Model
ECON 48 / WH Hog The Smple Regresso Model. Defto of the Smple Regresso Model Smple Regresso Model Expla varable y terms of varable x y = β + β x+ u y : depedet varable, explaed varable, respose varable,
f X (x i ;θ) = n ( n logx i = 0 = θml = n/ n logx i 1 θ +1 n n 2 < 0 for all θ (θ +1) n logx i 1 ESTIMATOR: = logx i θ n for all θ θ 2 < 0 2θ 2 x 3
MATH 557 - EXERCISES SOLUTIONS 1 The lkelhoo the orgal parameterzato s ( ( 1 L (θ 1,θ x 1,x = θ x 1 1 x (1 θ 1 1 x 1 θ x 1 x (1 θ x If φ = θ /(1 θ, the θ = φ/(1+φ θ 1 = (φψ/(1+φψ. Ether by wrtg out the
STA 108 Applied Linear Models: Regression Analysis Spring Solution for Homework #1
STA 08 Appled Lear Models: Regresso Aalyss Sprg 0 Soluto for Homework #. Let Y the dollar cost per year, X the umber of vsts per year. The the mathematcal relato betwee X ad Y s: Y 300 + X. Ths s a fuctoal
Unique Common Fixed Point of Sequences of Mappings in G-Metric Space M. Akram *, Nosheen
Vol No : Joural of Facult of Egeerg & echolog JFE Pages 9- Uque Coo Fed Pot of Sequeces of Mags -Metrc Sace M. Ara * Noshee * Deartet of Matheatcs C Uverst Lahore Pasta. Eal: ara7@ahoo.co Deartet of Matheatcs
Research on PCA and KPCA Self-Fusion Based MSTAR SAR Automatic Target Recognition Algorithm
35 JOURNAL OF ELERONI SIENE AND EHNOLOGY, VOL 0, NO 4, DEEMBER 0 Research o PA a KPA Self-Fuso Base MSAR SAR Autoatc arget Recogto Algorth huag L Fe Peg, Bg-Hu Wag, We-Feg Su, a Xag-Je Kog Abstract hs
A CLASS OF SINGULAR PERTURBATED BILOCAL LINEAR PROBLEMS
Proeegs of the Iteratoal Coferee o Theor a Applatos of Matheats a Iforats ICTAMI 3 Alba Iula A CLASS OF SINGULAR PERTURBATED BILOCAL LINEAR PROBLEMS b Mhaela Jaraat a Teoor Groşa Abstrat. Ths paper presets
Linear models for classification
CS 75 Mache Lear Lecture 9 Lear modes for cassfcato Mos Hausrecht mos@cs.ptt.edu 539 Seott Square ata: { d d.. d} d Cassfcato represets a dscrete cass vaue Goa: ear f : X Y Bar cassfcato A speca case he
Chapter Two. An Introduction to Regression ( )
ubject: A Itroducto to Regresso Frst tage Chapter Two A Itroducto to Regresso (018-019) 1 pg. ubject: A Itroducto to Regresso Frst tage A Itroducto to Regresso Regresso aalss s a statstcal tool for the
Midterm Exam 1, section 1 (Solution) Thursday, February hour, 15 minutes
coometrcs, CON Sa Fracsco State Uversty Mchael Bar Sprg 5 Mdterm am, secto Soluto Thursday, February 6 hour, 5 mutes Name: Istructos. Ths s closed book, closed otes eam.. No calculators of ay kd are allowed..
A New Method for Solving Fuzzy Linear. Programming by Solving Linear Programming
ppled Matheatcal Sceces Vol 008 o 50 7-80 New Method for Solvg Fuzzy Lear Prograg by Solvg Lear Prograg S H Nasser a Departet of Matheatcs Faculty of Basc Sceces Mazadara Uversty Babolsar Ira b The Research
SEMI-TIED FULL-COVARIANCE MATRICES FOR HMMS
SEMI-TIED FULL-COVARIANCE MATRICES FOR HMMS M.J.F. Gales fg@eg.ca.ac.uk Deceber 9, 997 Cotets Bass. Block Dagoal Matrces : : : : : : : : : : : : : : : : : : : : : : : : : : : :. Cooly used atrx dervatve
Objectives of Multiple Regression
Obectves of Multple Regresso Establsh the lear equato that best predcts values of a depedet varable Y usg more tha oe eplaator varable from a large set of potetal predctors {,,... k }. Fd that subset of
= 2. Statistic - function that doesn't depend on any of the known parameters; examples:
of Samplg Theory amples - uemploymet househol cosumpto survey Raom sample - set of rv's... ; 's have ot strbuto [ ] f f s vector of parameters e.g. Statstc - fucto that oes't epe o ay of the ow parameters;
LINEARLY CONSTRAINED MINIMIZATION BY USING NEWTON S METHOD
Jural Karya Asl Loreka Ahl Matematk Vol 8 o 205 Page 084-088 Jural Karya Asl Loreka Ahl Matematk LIEARLY COSTRAIED MIIMIZATIO BY USIG EWTO S METHOD Yosza B Dasrl, a Ismal B Moh 2 Faculty Electrocs a Computer
An Algorithm for Capacitated n-index Transportation Problem
Iteratoal Joural of Coutatoal Scece a Matheatcs ISSN 974-389 Volue 3, Nuber 3 2), 269-275 Iteratoal Research Publcato House htt://wwwrhouseco A Algorth for Caactate -Ie Trasortato Proble SC Shara a 2 Abha
Capacitated Plant Location Problem:
. L. Brcker, 2002 ept of Mechacal & Idustral Egeerg The Uversty of Iowa CPL/ 5/29/2002 page CPL/ 5/29/2002 page 2 Capactated Plat Locato Proble: where Mze F + C subect to = = =, =, S, =,... 0, =, ; =,
Useful Statistical Identities, Inequalities and Manipulations
Useful Statstcal Iettes, Iequaltes a Mapulatos Cotets Itroucto Probablty Os, Log-Os PA ( ) π Let π PA ( ) We efe the Os Rato θ If θ we say that the patet s three PA ( ) π tmes more lely to have the sease
Machine Learning. Introduction to Regression. Le Song. CSE6740/CS7641/ISYE6740, Fall 2012
Mache Learg CSE6740/CS764/ISYE6740, Fall 0 Itroducto to Regresso Le Sog Lecture 4, August 30, 0 Based o sldes from Erc g, CMU Readg: Chap. 3, CB Mache learg for apartmet hutg Suppose ou are to move to
Recall MLR 5 Homskedasticity error u has the same variance given any values of the explanatory variables Var(u x1,...,xk) = 2 or E(UU ) = 2 I
Chapter 8 Heterosedastcty Recall MLR 5 Homsedastcty error u has the same varace gve ay values of the eplaatory varables Varu,..., = or EUU = I Suppose other GM assumptos hold but have heterosedastcty.
Source-Channel-Sink Some questions
Source-Channel-Snk Soe questons Source Channel Snk Aount of Inforaton avalable Source Entro Generall nos and a be te varng Introduces error and lts the rate at whch data can be transferred ow uch nforaton
15-381: Artificial Intelligence. Regression and neural networks (NN)
5-38: Artfcal Itellece Reresso ad eural etorks NN) Mmck the bra I the earl das of AI there as a lot of terest develop models that ca mmc huma thk. Whle o oe ke eactl ho the bra orks ad, eve thouh there
Department of Mathematics UNIVERSITY OF OSLO. FORMULAS FOR STK4040 (version 1, September 12th, 2011) A - Vectors and matrices
Deartet of Matheatcs UNIVERSITY OF OSLO FORMULAS FOR STK4040 (verso Seteber th 0) A - Vectors ad atrces A) For a x atrx A ad a x atrx B we have ( AB) BA A) For osgular square atrces A ad B we have ( )
Training Sample Model: Given n observations, [[( Yi, x i the sample model can be expressed as (1) where, zero and variance σ
Stat 74 Estmato for Geeral Lear Model Prof. Goel Broad Outle Geeral Lear Model (GLM): Trag Samle Model: Gve observatos, [[( Y, x ), x = ( x,, xr )], =,,, the samle model ca be exressed as Y = µ ( x, x,,
Simple Linear Regression
Statstcal Methods I (EST 75) Page 139 Smple Lear Regresso Smple regresso applcatos are used to ft a model descrbg a lear relatoshp betwee two varables. The aspects of least squares regresso ad correlato
ε. Therefore, the estimate
Suggested Aswers, Problem Set 3 ECON 333 Da Hugerma. Ths s ot a very good dea. We kow from the secod FOC problem b) that ( ) SSE / = y x x = ( ) Whch ca be reduced to read y x x = ε x = ( ) The OLS model
ENGI 3423 Simple Linear Regression Page 12-01
ENGI 343 mple Lear Regresso Page - mple Lear Regresso ometmes a expermet s set up where the expermeter has cotrol over the values of oe or more varables X ad measures the resultg values of aother varable
Solving optimal margin classifier
Solvg optal arg classfer Recall our opt proble: s s equvalet to Wrte te Lagraga: Recall tat * ca be reforulated as No e solve ts dual proble: b b + s.t a b b + s.t 0 [ ] + b b L * a b b L 0 a b b L 0 ***
Artificial Intelligence Learning of decision trees
Artfcal Itellgece Learg of decso trees Peter Atal atal@mt.bme.hu A.I. November 21, 2016 1 Problem: decde whether to wat for a table at a restaurat, based o the followg attrbutes: 1. Alterate: s there a
Tomoki Toda. Augmented Human Communication Laboratory Graduate School of Information Science
Seuetial Data Modelig d class Basics of seuetial data odelig ooki oda Augeted Hua Couicatio Laboratory Graduate School of Iforatio Sciece Basic Aroaches How to efficietly odel joit robability of high diesioal
ECON 5360 Class Notes GMM
ECON 560 Class Notes GMM Geeralzed Method of Momets (GMM) I beg by outlg the classcal method of momets techque (Fsher, 95) ad the proceed to geeralzed method of momets (Hase, 98).. radtoal Method of Momets
12.2 Estimating Model parameters Assumptions: ox and y are related according to the simple linear regression model
1. Estmatg Model parameters Assumptos: ox ad y are related accordg to the smple lear regresso model (The lear regresso model s the model that says that x ad y are related a lear fasho, but the observed
3.1 Introduction to Multinomial Logit and Probit
ES3008 Ecooetrcs Lecture 3 robt ad Logt - Multoal 3. Itroducto to Multoal Logt ad robt 3. Estato of β 3. Itroducto to Multoal Logt ad robt The ultoal Logt odel s used whe there are several optos (ad therefore
Ordinary Least Squares Regression. Simple Regression. Algebra and Assumptions.
Ordary Least Squares egresso. Smple egresso. Algebra ad Assumptos. I ths part of the course we are gog to study a techque for aalysg the lear relatoshp betwee two varables Y ad X. We have pars of observatos
residual. (Note that usually in descriptions of regression analysis, upper-case
Regresso Aalyss Regresso aalyss fts or derves a model that descres the varato of a respose (or depedet ) varale as a fucto of oe or more predctor (or depedet ) varales. The geeral regresso model s oe of
Chapter Business Statistics: A First Course Fifth Edition. Learning Objectives. Correlation vs. Regression. In this chapter, you learn:
Chapter 3 3- Busess Statstcs: A Frst Course Ffth Edto Chapter 2 Correlato ad Smple Lear Regresso Busess Statstcs: A Frst Course, 5e 29 Pretce-Hall, Ic. Chap 2- Learg Objectves I ths chapter, you lear:
Solutions to problem set ); (, ) (
Solutos to proble set.. L = ( yp p ); L = ( p p ); y y L, L = yp p, p p = yp p, + p [, p ] y y y = yp + p = L y Here we use for eaple that yp, p = yp p p yp = yp, p = yp : factors that coute ca be treated
Uniform DFT Filter Banks 1/27
.. Ufor FT Flter Baks /27 Ufor FT Flter Baks We ll look at 5 versos of FT-based flter baks all but the last two have serous ltatos ad are t practcal. But they gve a ce trasto to the last two versos whch
Kernel-based Methods and Support Vector Machines
Kerel-based Methods ad Support Vector Maches Larr Holder CptS 570 Mache Learg School of Electrcal Egeerg ad Computer Scece Washgto State Uverst Refereces Muller et al. A Itroducto to Kerel-Based Learg
STA302/1001-Fall 2008 Midterm Test October 21, 2008
STA3/-Fall 8 Mdterm Test October, 8 Last Name: Frst Name: Studet Number: Erolled (Crcle oe) STA3 STA INSTRUCTIONS Tme allowed: hour 45 mutes Ads allowed: A o-programmable calculator A table of values from
Part I: Background on the Binomial Distribution
Part I: Bacgroud o the Bomal Dstrbuto A radom varable s sad to have a Beroull dstrbuto f t taes o the value wth probablt "p" ad the value wth probablt " - p". The umber of "successes" "" depedet Beroull
THE PUBLISHING HOUSE PROCEEDINGS OF THE ROMANIAN ACADEMY, Series A, OF THE ROMANIAN ACADEMY Volume 9, Number 3/2008, pp
THE PUBLISHIN HOUSE PROCEEDINS OF THE ROMANIAN ACADEMY, Seres A, OF THE ROMANIAN ACADEMY Volume 9, Number 3/8, THE UNITS IN Stela Corelu ANDRONESCU Uversty of Pteşt, Deartmet of Mathematcs, Târgu Vale
D. L. Bricker, 2002 Dept of Mechanical & Industrial Engineering The University of Iowa. CPL/XD 12/10/2003 page 1
D. L. Brcker, 2002 Dept of Mechacal & Idustral Egeerg The Uversty of Iowa CPL/XD 2/0/2003 page Capactated Plat Locato Proble: Mze FY + C X subject to = = j= where Y = j= X D, j =, j X SY, =,... X 0, =,
Feature Selection: Part 2. 1 Greedy Algorithms (continued from the last lecture)
CSE 546: Mache Learg Lecture 6 Feature Selecto: Part 2 Istructor: Sham Kakade Greedy Algorthms (cotued from the last lecture) There are varety of greedy algorthms ad umerous amg covetos for these algorthms.
-Pareto Optimality for Nondifferentiable Multiobjective Programming via Penalty Function
JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS 198, 248261 1996 ARTICLE NO. 0080 -Pareto Otalty for Nodfferetable Multobectve Prograg va Pealty Fucto J. C. Lu Secto of Matheatcs, Natoal Uersty Prearatory
Advances in Theory of Neural Network and Its Application
Joural of Behavoral a Bra Scece 06 6 9-6 Publshe Ole May 06 ScRes http://wwwscrporg/oural/bbs http://xoorg/0436/bbs06650 Avaces Theory of Neural Network a Its Applcato Baha Mashoo Greg Mllbak 50 a Playa
Dr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Aalyss of Varace ad Desg of Exermets-I MODULE II LECTURE - GENERAL LINEAR HYPOTHESIS AND ANALYSIS OF VARIANCE Dr Shalabh Deartmet of Mathematcs ad Statstcs Ida Isttute of Techology Kaur Tukey s rocedure
Multivariate Transformation of Variables and Maximum Likelihood Estimation
Marquette Uversty Multvarate Trasformato of Varables ad Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Assocate Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 03 by Marquette Uversty
Coherent Potential Approximation
Coheret Potetal Approxato Noveber 29, 2009 Gree-fucto atrces the TB forals I the tght bdg TB pcture the atrx of a Haltoa H s the for H = { H j}, where H j = δ j ε + γ j. 2 Sgle ad double uderles deote
New Trade Theory (1979)
Ne Trade Theory 979 Ne Trade Theory Krugma, 979: - Ecoomes of scale as reaso for trade - Elas trade betee smlar coutres Ituto of model: There s a trade-off betee ecoomes of scale the roducto of good tyes
Lecture 8: Linear Regression
Lecture 8: Lear egresso May 4, GENOME 56, Sprg Goals Develop basc cocepts of lear regresso from a probablstc framework Estmatg parameters ad hypothess testg wth lear models Lear regresso Su I Lee, CSE
Naïve Bayes MIT Course Notes Cynthia Rudin
Thaks to Şeyda Ertek Credt: Ng, Mtchell Naïve Bayes MIT 5.097 Course Notes Cytha Rud The Naïve Bayes algorthm comes from a geeratve model. There s a mportat dstcto betwee geeratve ad dscrmatve models.
Non-degenerate Perturbation Theory
No-degeerate Perturbato Theory Proble : H E ca't solve exactly. But wth H H H' H" L H E Uperturbed egevalue proble. Ca solve exactly. E Therefore, kow ad. H ' H" called perturbatos Copyrght Mchael D. Fayer,
Differential Encoding
Dfferetal Ecog C.M. Lu Perceptual Sgal Processg Lab College of Computer Scece Natoal Chao-Tug Uversty http://www.cse.ctu.eu.tw/~cmlu/courses/compresso/ Offce: EC538 (03)573877 cmlu@cs.ctu.eu.tw Iea eucg
Parameter Estimation
arameter Estmato robabltes Notatoal Coveto Mass dscrete fucto: catal letters Desty cotuous fucto: small letters Vector vs. scalar Scalar: la Vector: bold D: small Hgher dmeso: catal Notes a cotuous state
Linear Regression with One Regressor
Lear Regresso wth Oe Regressor AIM QA.7. Expla how regresso aalyss ecoometrcs measures the relatoshp betwee depedet ad depedet varables. A regresso aalyss has the goal of measurg how chages oe varable,