Some Different Perspectives on Linear Least Squares
|
|
- Moses Little
- 5 years ago
- Views:
Transcription
1 Soe Dfferet Perspectves o Lear Least Squares A stadard proble statstcs s to easure a respose or depedet varable, y, at fed values of oe or ore depedet varables. Soetes there ests a deterstc odel y f (,, where stads collectvely for all of the depedet varables of the syste ad stads collectvely for all of the odel paraeters. I the probablstc odel t s ofte assued that each easured y value ca be epressed as y f (, + ε, here s a de that labels the fed put codto at adε represets rado error assocated wth the gve easureet. For a vald odel the ea rado error should be zero. For a gve set of data pots the proble of choosg the best values for the paraeters s typcally solved va the ethod of least squares. he paraeters are chose to ze the fucto F( y f (,. If the fucto f (, s lear the procedure s referred to as a lear least squares or lear regresso proble. he purpose of these otes s to eae the least squares soluto fro several dfferet pots of vew. he Least Squares Approato to a Vector a Desoal Eucldea Space Cosder a desoal Eucldea space, V wth a er product of vectors v ad u deoted as( uv,. Let W be a subspace of V, the V W W, where W s the orthogoal copleet of W. Let v be ay eleet of V, thev w+ u, where w W ad u W. he vector w s called the proecto of v oto W. Let g v ( ψ v ψ ( v ψ, v ψ. he followg arguet shows that ψ w s the g v ψ ca be epressed as g v ( ψ ( v w ( ψ w v w ( v w, ψ w + ψ w. If ψ W, the ψ w W ad ( v w, ψ w ( u, ψ w 0. So o W, g v ( ψ u + ψ w u, wth equalty oly at ψ w. If V R ad W has the orthooral bass C, C, C wth, the the colu vector uque u of ths fucto o W. Frst, by epadg the er product, ( { } whch s the proecto of v oto W s gve by w ( Cˆ, v Cˆ (. Defe the atr Q as Q C, C, C, that s Q, C. he atr QQ, called the outer product,, proects v oto W: ( ( ( w, C, v C C ( Q v, Q, ( Q v, ( QQ v. hus,,,, the vector QQ v zes the squared dstace fro v to W. Al Lehe 9/4/0 Madso Area echcal College
2 Soe Dfferet Perspectves o Lear Least Squares Lear Least Squares Matr Notato: he Geeralzed Iverse If a odel s lear ts paraeters desgated by the vector R, the all the estated y resposes ca be geerated as a vector R gve by A where A s a atr. We wll assue that A has ra wth. If the ra of A s less tha, t eas that the odel paraeters are ot all depedet ad eed to be reduced uber so as to ae a desg atr wth all of ts colus learly depedet. Deote the th colu of A as A,.e., A [ A, A, A ]. Sce the colus of A are learly depedet vectors R, Ab ba 0 f ad oly f each b 0. hg of A as a lear trasforato fro R to R, the age of A, W, s the spa of { A, A, A } ad has deso, whle the erel of A has deso zero ad cossts of oly the zero vector 0. hus, every vector W s orthogoal to each colu of A, or stated dfferetly, f u W, A u 0. Coversely, f A u 0, the every colu of A s perpedcular to u, so u W. hus, the erel of A traspose s the orthogoal copleet of the colu space of A. Let y R, the y w + uwhere w W ad u W. Uless u 0, the syste of equatos Ab y has o solutos. However, Ab w has a soluto,, ad sce { A, A, A } s a learly depedet set, ths soluto s uque. Furtherore, w s the closest vector the colu space of A to y ad the u value of F y ( b y Ab as b vares over R s F y ( y w y A. Now, A w, so, sce er( A A A A w A ( y u A y A u A y W. Suppose that for b R, A Ab 0, the b A Ab ( Ab Ab ( Ab, Ab Ab 0. So, f A Ab 0, Ab 0 whch ples that b 0 sce the erel of A cossts of oly 0. herefore, A As osgular ad the least squares soluto to zg F y ( b s gve by ( A A A y, the atr ( A A A s soetes called the geeralzed verse of the atr A. A Alterate Forulato of Lear Least Squares Usg Multvarable Calculus he least squares soluto s the vector R that zes the fucto of varables, F y ( b y Ab ( yab, y Ab ( y, y ( Ab, y + ( Ab, Ab as b vares over R. Rewrtg ths fucto usg sga otato gves the followg epresso. Al Lehe 9/4/0 Madso Area echcal College
3 Soe Dfferet Perspectves o Lear Least Squares 3 F y ( b ( y, y ( Ab y, + A, b b A, b ( y, y ( A by + b b( A A,, Now a ecessary codto for F y ( b to be at a u s that all of the partal dervatves F y b vash for every teger wth. Sce δ,, the followg splfcato results. b b F y ( yy, ( A b, y + b b ( A A b b, b 0 ( b b A y, + b b ( A A + b b b, ( A y, + ( δ, b b δ, ( A A +, ( A y + ( A A b + ( A A b,,, ( A y + ( A Ab + ( A A b,,, ( A y + ( A Ab,, F y he codto, 0 b uque soluto gve by ( for every teger wth, requres that A Ab A y. hs has the A A A y, whch s detcal to the result obtaed the last secto. A Forulato of Lear Least Squares Usg QR Factorzato Sce the colus of A are learly depedet a Gra Schdt procedure o the colus yelds a orthooral bass for W. Desgate ths bass as C, C, C. Specfcally, ths orthooral bass s { } defed recursvely for as C C, where C A ( A, C C C ad C A. A Al Lehe 9/4/0 Madso Area echcal College
4 Soe Dfferet Perspectves o Lear Least Squares 4 he colu vector whch s the proecto of y oto W s gve by w QQ y, where the atr Q s defed as Q (, C,.e., Q C, C, C. Now, A QQ A, sce the proecto of each, colu of atr A oto A s colu space s ust that orgal colu of A. Let the atr R be defed as R Q A. Now, ( ( R C (,, A C A. By the Gra Schdt procedure f the spa of { C, C, C } s equal to the spa of { A, A, A }. So A ( C C, A. Now, f >, ( ( ( C, A C, C C, A ( 0 ( C, A 0,.e., f >, C s orthogoal to the subspace spaed by { A, A, A }. herefore, the atr R s upper tragular. Cosder for real ubers, α,, α,, α3,, α, the su α (, ( R, α C A C, z, where z α A W. If α R, 0 for every teger,, the z ( C, z C 0. But sce the colus of A are learly depedet, ths requres that α α α3 α 0. Hece, the colus of the atr R are learly depedet ad R s therefore osgular. hs factorzato of A, A QQ A QR, s geercally called the QR factorzato. Sce the uque soluto of the lear least squares proble solves A w QQ y, we have QR QQ y. Now, Q Q I,.e., ( (,, (, Q Q Q C C C C C C C δ,.,, So R Q QR Q QQ y Q yad the uque lear least squares soluto s gve by R Q y. he followg steps show that ths soluto s the sae as that obtaed by the geeralzed verse. A ( QR R Q A A R Q QR R IR R R A A R R R R ( ( ( ( ( A A A R R R Q R Q Calculatg the Paraeter Varaces Fro the theory of the probablty dstrbutos of rado varables we have the followg fudaetal result. If y, y, y3,, y are statstcally depedet rado varables wth the varace of y beg Al Lehe 9/4/0 Madso Area echcal College
5 Soe Dfferet Perspectves o Lear Least Squares 5 σ, ad h α y, the σh α σ. Now, usg the geeralzed verse the lear least squares soluto for paraeter s gve by ( A A A y ( A A A y y A, ( A A.,,, Hece, ( σ σ A A A.,, I the specal case where all of the rado varables have a coo varace, σ, ths splfes to ( ( σ σ A A A σ A A A,.,, I the QR factorzato soluto of the lear least squares proble, ( R Q y ( R Q y y R (, Q, y R, C, so that,, σ σ R, ( C. I the specal case where all of the rado varables have a coo varace ths splfes as follows. ( ( ( R ( (, C R, C R, C R, R, C C σ R, R, ( C, C σ R, R, δ, σ ( R, A Eaple of the Methods: he Lear Model Oe Idepedet Varable If the odel s that yestate + 0, ad the paraeter vector s defed as, the the 0 desg atr gve by A. Hece, A A ad Al Lehe 9/4/0 Madso Area echcal College
6 Soe Dfferet Perspectves o Lear Least Squares 6 ( A A. Here the average value of a varable s deoted as α α ( whle the covarato of two varables αω, s defed as α ω αω + αω α ω ( αω( α( ω ( α α( ω ω αω ω α ( α( ω αω ( α( ω y Also, A y, so the least squares soluto s gve by the followg epresso, y ( y y y A A A y. 0 y y y hese are the regresso equatos whch are soetes epressed as y y y y y 0 y y y y y y y y ( y ( y Assug a coo varace for each depedet rado varable y, the varaces of the odel paraeters are coputed as follows.. Al Lehe 9/4/0 Madso Area echcal College
7 Soe Dfferet Perspectves o Lear Least Squares 7 σ σ σ ( A A A (, σ σ σ σ σ σ ( σ ( A A A ( 0, σ ( (( ( σ ( ( ( ( + ( ( σ ( ( ( ( ( ( + ( σ σ [ ] ( ( ( 0 + ( + ( ( σ + Usg the QR forulato to calculate the paraeters requres a orthooral bass for the colu space of A. A Gra Schdt procedure results the followg vectors. C, C Al Lehe 9/4/0 Madso Area echcal College
8 Soe Dfferet Perspectves o Lear Least Squares 8 Noralzg C requres the prevously ecoutered su, ( ( + (. So that ( C. ( + ( hus, Q + ad ( ( C, A C, A R Q A ( C, A ( 0 0. Proceedg, C, A ( ( + + ( ( + + ( R + ad ( y Q y. y y ( + Coputg the paraeter vector gves y y y R Q y. 0 y y Al Lehe 9/4/0 Madso Area echcal College
9 Soe Dfferet Perspectves o Lear Least Squares 9 he epresso for 0 agrees wth the earler dervato, whle the prevous epresso for s recovered as s show by the followg steps. y y y y y y y y y y y y y y y Assug a coo varace for each depedet rado varable y, the QR factorzato the varaces of the odel paraeters are coputed as follows. ( σ ( ( R, ( σ 4 σ 4 ( ( ( ( ( σ σ ( ( ( R, σ σ σ σ σ σ σ Both of these results agree wth the prevous aalyss that used the geeralzed verse. Al Lehe 9/4/0 Madso Area echcal College
7.0 Equality Contraints: Lagrange Multipliers
Systes Optzato 7.0 Equalty Cotrats: Lagrage Multplers Cosder the zato of a o-lear fucto subject to equalty costrats: g f() R ( ) 0 ( ) (7.) where the g ( ) are possbly also olear fuctos, ad < otherwse
More informationA New Method for Solving Fuzzy Linear. Programming by Solving Linear Programming
ppled Matheatcal Sceces Vol 008 o 50 7-80 New Method for Solvg Fuzzy Lear Prograg by Solvg Lear Prograg S H Nasser a Departet of Matheatcs Faculty of Basc Sceces Mazadara Uversty Babolsar Ira b The Research
More information3D Reconstruction from Image Pairs. Reconstruction from Multiple Views. Computing Scene Point from Two Matching Image Points
D Recostructo fro Iage ars Recostructo fro ultple Ves Dael Deetho Fd terest pots atch terest pots Copute fudaetal atr F Copute caera atrces ad fro F For each atchg age pots ad copute pot scee Coputg Scee
More informationCS 2750 Machine Learning. Lecture 7. Linear regression. CS 2750 Machine Learning. Linear regression. is a linear combination of input components x
CS 75 Mache Learg Lecture 7 Lear regresso Mlos Hauskrecht los@cs.ptt.edu 59 Seott Square CS 75 Mache Learg Lear regresso Fucto f : X Y s a lear cobato of put copoets f + + + K d d K k - paraeters eghts
More informationSolutions to problem set ); (, ) (
Solutos to proble set.. L = ( yp p ); L = ( p p ); y y L, L = yp p, p p = yp p, + p [, p ] y y y = yp + p = L y Here we use for eaple that yp, p = yp p p yp = yp, p = yp : factors that coute ca be treated
More informationKURODA S METHOD FOR CONSTRUCTING CONSISTENT INPUT-OUTPUT DATA SETS. Peter J. Wilcoxen. Impact Research Centre, University of Melbourne.
KURODA S METHOD FOR CONSTRUCTING CONSISTENT INPUT-OUTPUT DATA SETS by Peter J. Wlcoxe Ipact Research Cetre, Uversty of Melboure Aprl 1989 Ths paper descrbes a ethod that ca be used to resolve cossteces
More informationA Conventional Approach for the Solution of the Fifth Order Boundary Value Problems Using Sixth Degree Spline Functions
Appled Matheatcs, 1, 4, 8-88 http://d.do.org/1.4/a.1.448 Publshed Ole Aprl 1 (http://www.scrp.org/joural/a) A Covetoal Approach for the Soluto of the Ffth Order Boudary Value Probles Usg Sth Degree Sple
More informationChapter 5 Properties of a Random Sample
Lecture 6 o BST 63: Statstcal Theory I Ku Zhag, /0/008 Revew for the prevous lecture Cocepts: t-dstrbuto, F-dstrbuto Theorems: Dstrbutos of sample mea ad sample varace, relatoshp betwee sample mea ad sample
More informationSome results and conjectures about recurrence relations for certain sequences of binomial sums.
Soe results ad coectures about recurrece relatos for certa sequeces of boal sus Joha Cgler Faultät für Matheat Uverstät We A-9 We Nordbergstraße 5 Joha Cgler@uveacat Abstract I a prevous paper [] I have
More informationLecture Notes 2. The ability to manipulate matrices is critical in economics.
Lecture Notes. Revew of Matrces he ablt to mapulate matrces s crtcal ecoomcs.. Matr a rectagular arra of umbers, parameters, or varables placed rows ad colums. Matrces are assocated wth lear equatos. lemets
More informationNon-degenerate Perturbation Theory
No-degeerate Perturbato Theory Proble : H E ca't solve exactly. But wth H H H' H" L H E Uperturbed egevalue proble. Ca solve exactly. E Therefore, kow ad. H ' H" called perturbatos Copyrght Mchael D. Fayer,
More information( ) ( ) ( ( )) ( ) ( ) ( ) ( ) ( ) = ( ) ( ) + ( ) ( ) = ( ( )) ( ) + ( ( )) ( ) Review. Second Derivatives for f : y R. Let A be an m n matrix.
Revew + v, + y = v, + v, + y, + y, Cato! v, + y, + v, + y geeral Let A be a atr Let f,g : Ω R ( ) ( ) R y R Ω R h( ) f ( ) g ( ) ( ) ( ) ( ( )) ( ) dh = f dg + g df A, y y A Ay = = r= c= =, : Ω R he Proof
More informationLecture 3 Probability review (cont d)
STATS 00: Itroducto to Statstcal Iferece Autum 06 Lecture 3 Probablty revew (cot d) 3. Jot dstrbutos If radom varables X,..., X k are depedet, the ther dstrbuto may be specfed by specfyg the dvdual dstrbuto
More informationare positive, and the pair A, B is controllable. The uncertainty in is introduced to model control failures.
Lectue 4 8. MRAC Desg fo Affe--Cotol MIMO Systes I ths secto, we cosde MRAC desg fo a class of ult-ut-ult-outut (MIMO) olea systes, whose lat dyacs ae lealy aaetezed, the ucetates satsfy the so-called
More informationThe Mathematical Appendix
The Mathematcal Appedx Defto A: If ( Λ, Ω, where ( λ λ λ whch the probablty dstrbutos,,..., Defto A. uppose that ( Λ,,..., s a expermet type, the σ-algebra o λ λ λ are defed s deoted by ( (,,...,, σ Ω.
More informationAlgorithms behind the Correlation Setting Window
Algorths behd the Correlato Settg Wdow Itroducto I ths report detaled forato about the correlato settg pop up wdow s gve. See Fgure. Ths wdow s obtaed b clckg o the rado butto labelled Kow dep the a scree
More informationSpecial Instructions / Useful Data
JAM 6 Set of all real umbers P A..d. B, p Posso Specal Istructos / Useful Data x,, :,,, x x Probablty of a evet A Idepedetly ad detcally dstrbuted Bomal dstrbuto wth parameters ad p Posso dstrbuto wth
More informationThe Mathematics of Portfolio Theory
The Matheatcs of Portfolo Theory The rates of retur of stocks, ad are as follows Market odtos state / scearo) earsh Neutral ullsh Probablty 0. 0.5 0.3 % 5% 9% -3% 3% % 5% % -% Notato: R The retur of stock
More informationPRACTICAL CONSIDERATIONS IN HUMAN-INDUCED VIBRATION
PRACTICAL CONSIDERATIONS IN HUMAN-INDUCED VIBRATION Bars Erkus, 4 March 007 Itroducto Ths docuet provdes a revew of fudaetal cocepts structural dyacs ad soe applcatos hua-duced vbrato aalyss ad tgato of
More informationFunctions of Random Variables
Fuctos of Radom Varables Chapter Fve Fuctos of Radom Varables 5. Itroducto A geeral egeerg aalyss model s show Fg. 5.. The model output (respose) cotas the performaces of a system or product, such as weght,
More informationDr. Shalabh. Indian Institute of Technology Kanpur
Aalyss of Varace ad Desg of Expermets-I MODULE -I LECTURE - SOME RESULTS ON LINEAR ALGEBRA, MATRIX THEORY AND DISTRIBUTIONS Dr. Shalabh Departmet t of Mathematcs t ad Statstcs t t Ida Isttute of Techology
More informationDepartment of Mathematics UNIVERSITY OF OSLO. FORMULAS FOR STK4040 (version 1, September 12th, 2011) A - Vectors and matrices
Deartet of Matheatcs UNIVERSITY OF OSLO FORMULAS FOR STK4040 (verso Seteber th 0) A - Vectors ad atrces A) For a x atrx A ad a x atrx B we have ( AB) BA A) For osgular square atrces A ad B we have ( )
More informationRelations to Other Statistical Methods Statistical Data Analysis with Positive Definite Kernels
Relatos to Other Statstcal Methods Statstcal Data Aalyss wth Postve Defte Kerels Kej Fukuzu Isttute of Statstcal Matheatcs, ROIS Departet of Statstcal Scece, Graduate Uversty for Advaced Studes October
More informationUNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Postpoed exam: ECON430 Statstcs Date of exam: Jauary 0, 0 Tme for exam: 09:00 a.m. :00 oo The problem set covers 5 pages Resources allowed: All wrtte ad prted
More informationOrdinary Least Squares Regression. Simple Regression. Algebra and Assumptions.
Ordary Least Squares egresso. Smple egresso. Algebra ad Assumptos. I ths part of the course we are gog to study a techque for aalysg the lear relatoshp betwee two varables Y ad X. We have pars of observatos
More informationIntroduction to Matrices and Matrix Approach to Simple Linear Regression
Itroducto to Matrces ad Matrx Approach to Smple Lear Regresso Matrces Defto: A matrx s a rectagular array of umbers or symbolc elemets I may applcatos, the rows of a matrx wll represet dvduals cases (people,
More informationD. L. Bricker, 2002 Dept of Mechanical & Industrial Engineering The University of Iowa. CPL/XD 12/10/2003 page 1
D. L. Brcker, 2002 Dept of Mechacal & Idustral Egeerg The Uversty of Iowa CPL/XD 2/0/2003 page Capactated Plat Locato Proble: Mze FY + C X subject to = = j= where Y = j= X D, j =, j X SY, =,... X 0, =,
More informationQualifying Exam Statistical Theory Problem Solutions August 2005
Qualfyg Exam Statstcal Theory Problem Solutos August 5. Let X, X,..., X be d uform U(,),
More informationCoherent Potential Approximation
Coheret Potetal Approxato Noveber 29, 2009 Gree-fucto atrces the TB forals I the tght bdg TB pcture the atrx of a Haltoa H s the for H = { H j}, where H j = δ j ε + γ j. 2 Sgle ad double uderles deote
More information2/20/2013. Topics. Power Flow Part 1 Text: Power Transmission. Power Transmission. Power Transmission. Power Transmission
/0/0 Topcs Power Flow Part Text: 0-0. Power Trassso Revsted Power Flow Equatos Power Flow Proble Stateet ECEGR 45 Power Systes Power Trassso Power Trassso Recall that for a short trassso le, the power
More informationMultivariate Transformation of Variables and Maximum Likelihood Estimation
Marquette Uversty Multvarate Trasformato of Varables ad Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Assocate Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 03 by Marquette Uversty
More informationDebabrata Dey and Atanu Lahiri
RESEARCH ARTICLE QUALITY COMPETITION AND MARKET SEGMENTATION IN THE SECURITY SOFTWARE MARKET Debabrata Dey ad Atau Lahr Mchael G. Foster School of Busess, Uersty of Washgto, Seattle, Seattle, WA 9895 U.S.A.
More informationX ε ) = 0, or equivalently, lim
Revew for the prevous lecture Cocepts: order statstcs Theorems: Dstrbutos of order statstcs Examples: How to get the dstrbuto of order statstcs Chapter 5 Propertes of a Radom Sample Secto 55 Covergece
More informationCHAPTER 4 RADICAL EXPRESSIONS
6 CHAPTER RADICAL EXPRESSIONS. The th Root of a Real Number A real umber a s called the th root of a real umber b f Thus, for example: s a square root of sce. s also a square root of sce ( ). s a cube
More informationStandard Deviation for PDG Mass Data
4 Dec 06 Stadard Devato for PDG Mass Data M. J. Gerusa Retred, 47 Clfde Road, Worghall, HP8 9JR, UK. gerusa@aol.co, phoe: +(44) 844 339754 Abstract Ths paper aalyses the data for the asses of eleetary
More informationMULTIOBJECTIVE NONLINEAR FRACTIONAL PROGRAMMING PROBLEMS INVOLVING GENERALIZED d - TYPE-I n -SET FUNCTIONS
THE PUBLIHING HOUE PROCEEDING OF THE ROMANIAN ACADEMY, eres A OF THE ROMANIAN ACADEMY Volue 8, Nuber /27,.- MULTIOBJECTIVE NONLINEAR FRACTIONAL PROGRAMMING PROBLEM INVOLVING GENERALIZED d - TYPE-I -ET
More informationSimple Linear Regression
Statstcal Methods I (EST 75) Page 139 Smple Lear Regresso Smple regresso applcatos are used to ft a model descrbg a lear relatoshp betwee two varables. The aspects of least squares regresso ad correlato
More informationTESTS BASED ON MAXIMUM LIKELIHOOD
ESE 5 Toy E. Smth. The Basc Example. TESTS BASED ON MAXIMUM LIKELIHOOD To llustrate the propertes of maxmum lkelhood estmates ad tests, we cosder the smplest possble case of estmatg the mea of the ormal
More informationMatrix Algebra Tutorial With Examples in Matlab
Matr Algebra Tutoral Wth Eamples Matlab by Klaus Moelter Departmet of Agrcultural ad Appled Ecoomcs Vrga Tech emal: moelter@vt.edu web: http://faculty.ageco.vt.edu/moelter/ Specfcally desged as a / day
More informationChapter 9 Jordan Block Matrices
Chapter 9 Jorda Block atrces I ths chapter we wll solve the followg problem. Gve a lear operator T fd a bass R of F such that the matrx R (T) s as smple as possble. f course smple s a matter of taste.
More informationAssignment 5/MATH 247/Winter Due: Friday, February 19 in class (!) (answers will be posted right after class)
Assgmet 5/MATH 7/Wter 00 Due: Frday, February 9 class (!) (aswers wll be posted rght after class) As usual, there are peces of text, before the questos [], [], themselves. Recall: For the quadratc form
More informationMATH 247/Winter Notes on the adjoint and on normal operators.
MATH 47/Wter 00 Notes o the adjot ad o ormal operators I these otes, V s a fte dmesoal er product space over, wth gve er * product uv, T, S, T, are lear operators o V U, W are subspaces of V Whe we say
More informationQR Factorization and Singular Value Decomposition COS 323
QR Factorzato ad Sgular Value Decomposto COS 33 Why Yet Aother Method? How do we solve least-squares wthout currg codto-squarg effect of ormal equatos (A T A A T b) whe A s sgular, fat, or otherwse poorly-specfed?
More informationChapter 14 Logistic Regression Models
Chapter 4 Logstc Regresso Models I the lear regresso model X β + ε, there are two types of varables explaatory varables X, X,, X k ad study varable y These varables ca be measured o a cotuous scale as
More informationDiscrete Mathematics and Probability Theory Fall 2016 Seshia and Walrand DIS 10b
CS 70 Dscrete Mathematcs ad Probablty Theory Fall 206 Sesha ad Walrad DIS 0b. Wll I Get My Package? Seaky delvery guy of some compay s out delverg packages to customers. Not oly does he had a radom package
More informationThe theoretical background of
he theoretcal backgroud of -echologes he theoretcal backgroud of FactSage he followg sldes gve a abrdged overvew of the ajor uderlyg prcples of the calculatoal odules of FactSage. -echologes he bbs Eergy
More informationChapter 2 - Free Vibration of Multi-Degree-of-Freedom Systems - II
CEE49b Chapter - Free Vbrato of Mult-Degree-of-Freedom Systems - II We ca obta a approxmate soluto to the fudametal atural frequecy through a approxmate formula developed usg eergy prcples by Lord Raylegh
More informationCapacitated Plant Location Problem:
. L. Brcker, 2002 ept of Mechacal & Idustral Egeerg The Uversty of Iowa CPL/ 5/29/2002 page CPL/ 5/29/2002 page 2 Capactated Plat Locato Proble: where Mze F + C subect to = = =, =, S, =,... 0, =, ; =,
More informationSTK4011 and STK9011 Autumn 2016
STK4 ad STK9 Autum 6 Pot estmato Covers (most of the followg materal from chapter 7: Secto 7.: pages 3-3 Secto 7..: pages 3-33 Secto 7..: pages 35-3 Secto 7..3: pages 34-35 Secto 7.3.: pages 33-33 Secto
More informationECONOMETRIC THEORY. MODULE VIII Lecture - 26 Heteroskedasticity
ECONOMETRIC THEORY MODULE VIII Lecture - 6 Heteroskedastcty Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur . Breusch Paga test Ths test ca be appled whe the replcated data
More informationBounds on the expected entropy and KL-divergence of sampled multinomial distributions. Brandon C. Roy
Bouds o the expected etropy ad KL-dvergece of sampled multomal dstrbutos Brado C. Roy bcroy@meda.mt.edu Orgal: May 18, 2011 Revsed: Jue 6, 2011 Abstract Iformato theoretc quattes calculated from a sampled
More information( ) = ( ) ( ) Chapter 13 Asymptotic Theory and Stochastic Regressors. Stochastic regressors model
Chapter 3 Asmptotc Theor ad Stochastc Regressors The ature of eplaator varable s assumed to be o-stochastc or fed repeated samples a regresso aalss Such a assumpto s approprate for those epermets whch
More information4 Inner Product Spaces
11.MH1 LINEAR ALGEBRA Summary Notes 4 Ier Product Spaces Ier product s the abstracto to geeral vector spaces of the famlar dea of the scalar product of two vectors or 3. I what follows, keep these key
More informationThe number of observed cases The number of parameters. ith case of the dichotomous dependent variable. the ith case of the jth parameter
LOGISTIC REGRESSION Notato Model Logstc regresso regresses a dchotomous depedet varable o a set of depedet varables. Several methods are mplemeted for selectg the depedet varables. The followg otato s
More informationLecture 8 IEEE DCF Performance
Lecture 8 IEEE82. DCF Perforace IEEE82. DCF Basc Access Mechas A stato wth a ew packet to trast otors the chael actvty. If the chael s dle for a perod of te equal to a dstrbuted terfrae space (DIFS), the
More informationGlobal Optimization for Solving Linear Non-Quadratic Optimal Control Problems
Joural of Appled Matheatcs ad Physcs 06 4 859-869 http://wwwscrporg/joural/jap ISSN Ole: 37-4379 ISSN Prt: 37-435 Global Optzato for Solvg Lear No-Quadratc Optal Cotrol Probles Jghao Zhu Departet of Appled
More informationPolyphase Filters. Section 12.4 Porat
Polyphase Flters Secto.4 Porat .4 Polyphase Flters Polyphase s a way of dog saplg-rate coverso that leads to very effcet pleetatos. But ore tha that, t leads to very geeral vewpots that are useful buldg
More informationRademacher Complexity. Examples
Algorthmc Foudatos of Learg Lecture 3 Rademacher Complexty. Examples Lecturer: Patrck Rebesch Verso: October 16th 018 3.1 Itroducto I the last lecture we troduced the oto of Rademacher complexty ad showed
More informationSEMI-TIED FULL-COVARIANCE MATRICES FOR HMMS
SEMI-TIED FULL-COVARIANCE MATRICES FOR HMMS M.J.F. Gales fg@eg.ca.ac.uk Deceber 9, 997 Cotets Bass. Block Dagoal Matrces : : : : : : : : : : : : : : : : : : : : : : : : : : : :. Cooly used atrx dervatve
More information1 Lyapunov Stability Theory
Lyapuov Stablty heory I ths secto we cosder proofs of stablty of equlbra of autoomous systems. hs s stadard theory for olear systems, ad oe of the most mportat tools the aalyss of olear systems. It may
More information5. Data Compression. Review of Last Lecture. Outline of the Lecture. Course Overview. Basics of Information Theory: Markku Juntti
: Markku Jutt Overvew The deas of lossless data copresso ad source codg are troduced ad copresso lts are derved. Source The ateral s aly based o Sectos 5. 5.5 of the course book []. Teleco. Laboratory
More informationESS Line Fitting
ESS 5 014 17. Le Fttg A very commo problem data aalyss s lookg for relatoshpetwee dfferet parameters ad fttg les or surfaces to data. The smplest example s fttg a straght le ad we wll dscuss that here
More informationEconometric Methods. Review of Estimation
Ecoometrc Methods Revew of Estmato Estmatg the populato mea Radom samplg Pot ad terval estmators Lear estmators Ubased estmators Lear Ubased Estmators (LUEs) Effcecy (mmum varace) ad Best Lear Ubased Estmators
More informationLecture Notes Types of economic variables
Lecture Notes 3 1. Types of ecoomc varables () Cotuous varable takes o a cotuum the sample space, such as all pots o a le or all real umbers Example: GDP, Polluto cocetrato, etc. () Dscrete varables fte
More informationSimulation Output Analysis
Smulato Output Aalyss Summary Examples Parameter Estmato Sample Mea ad Varace Pot ad Iterval Estmato ermatg ad o-ermatg Smulato Mea Square Errors Example: Sgle Server Queueg System x(t) S 4 S 4 S 3 S 5
More informationTransforms that are commonly used are separable
Trasforms s Trasforms that are commoly used are separable Eamples: Two-dmesoal DFT DCT DST adamard We ca the use -D trasforms computg the D separable trasforms: Take -D trasform of the rows > rows ( )
More informationSTATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. x, where. = y - ˆ " 1
STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS Recall Assumpto E(Y x) η 0 + η x (lear codtoal mea fucto) Data (x, y ), (x 2, y 2 ),, (x, y ) Least squares estmator ˆ E (Y x) ˆ " 0 + ˆ " x, where ˆ
More informationUnique Common Fixed Point of Sequences of Mappings in G-Metric Space M. Akram *, Nosheen
Vol No : Joural of Facult of Egeerg & echolog JFE Pages 9- Uque Coo Fed Pot of Sequeces of Mags -Metrc Sace M. Ara * Noshee * Deartet of Matheatcs C Uverst Lahore Pasta. Eal: ara7@ahoo.co Deartet of Matheatcs
More informationOrder Nonlinear Vector Differential Equations
It. Joural of Math. Aalyss Vol. 3 9 o. 3 39-56 Coverget Power Seres Solutos of Hgher Order Nolear Vector Dfferetal Equatos I. E. Kougas Departet of Telecoucato Systes ad Networs Techologcal Educatoal Isttute
More information18.413: Error Correcting Codes Lab March 2, Lecture 8
18.413: Error Correctg Codes Lab March 2, 2004 Lecturer: Dael A. Spelma Lecture 8 8.1 Vector Spaces A set C {0, 1} s a vector space f for x all C ad y C, x + y C, where we take addto to be compoet wse
More information6.867 Machine Learning
6.867 Mache Learg Problem set Due Frday, September 9, rectato Please address all questos ad commets about ths problem set to 6.867-staff@a.mt.edu. You do ot eed to use MATLAB for ths problem set though
More informationMultiple Choice Test. Chapter Adequacy of Models for Regression
Multple Choce Test Chapter 06.0 Adequac of Models for Regresso. For a lear regresso model to be cosdered adequate, the percetage of scaled resduals that eed to be the rage [-,] s greater tha or equal to
More information3.1 Introduction to Multinomial Logit and Probit
ES3008 Ecooetrcs Lecture 3 robt ad Logt - Multoal 3. Itroducto to Multoal Logt ad robt 3. Estato of β 3. Itroducto to Multoal Logt ad robt The ultoal Logt odel s used whe there are several optos (ad therefore
More informationSTK3100 and STK4100 Autumn 2018
SK3 ad SK4 Autum 8 Geeralzed lear models Part III Covers the followg materal from chaters 4 ad 5: Cofdece tervals by vertg tests Cosder a model wth a sgle arameter β We may obta a ( α% cofdece terval for
More information1 Solution to Problem 6.40
1 Soluto to Problem 6.40 (a We wll wrte T τ (X 1,...,X where the X s are..d. wth PDF f(x µ, σ 1 ( x µ σ g, σ where the locato parameter µ s ay real umber ad the scale parameter σ s > 0. Lettg Z X µ σ we
More informationChapter 13, Part A Analysis of Variance and Experimental Design. Introduction to Analysis of Variance. Introduction to Analysis of Variance
Chapter, Part A Aalyss of Varace ad Epermetal Desg Itroducto to Aalyss of Varace Aalyss of Varace: Testg for the Equalty of Populato Meas Multple Comparso Procedures Itroducto to Aalyss of Varace Aalyss
More informationObjectives of Multiple Regression
Obectves of Multple Regresso Establsh the lear equato that best predcts values of a depedet varable Y usg more tha oe eplaator varable from a large set of potetal predctors {,,... k }. Fd that subset of
More informationSTK3100 and STK4100 Autumn 2017
SK3 ad SK4 Autum 7 Geeralzed lear models Part III Covers the followg materal from chaters 4 ad 5: Sectos 4..5, 4.3.5, 4.3.6, 4.4., 4.4., ad 4.4.3 Sectos 5.., 5.., ad 5.5. Ørulf Borga Deartmet of Mathematcs
More information9 U-STATISTICS. Eh =(m!) 1 Eh(X (1),..., X (m ) ) i.i.d
9 U-STATISTICS Suppose,,..., are P P..d. wth CDF F. Our goal s to estmate the expectato t (P)=Eh(,,..., m ). Note that ths expectato requres more tha oe cotrast to E, E, or Eh( ). Oe example s E or P((,
More informationLecture 12 APPROXIMATION OF FIRST ORDER DERIVATIVES
FDM: Appromato of Frst Order Dervatves Lecture APPROXIMATION OF FIRST ORDER DERIVATIVES. INTRODUCTION Covectve term coservato equatos volve frst order dervatves. The smplest possble approach for dscretzato
More informationTraining Sample Model: Given n observations, [[( Yi, x i the sample model can be expressed as (1) where, zero and variance σ
Stat 74 Estmato for Geeral Lear Model Prof. Goel Broad Outle Geeral Lear Model (GLM): Trag Samle Model: Gve observatos, [[( Y, x ), x = ( x,, xr )], =,,, the samle model ca be exressed as Y = µ ( x, x,,
More informationHomework 1: Solutions Sid Banerjee Problem 1: (Practice with Asymptotic Notation) ORIE 4520: Stochastics at Scale Fall 2015
Fall 05 Homework : Solutos Problem : (Practce wth Asymptotc Notato) A essetal requremet for uderstadg scalg behavor s comfort wth asymptotc (or bg-o ) otato. I ths problem, you wll prove some basc facts
More informationECON 5360 Class Notes GMM
ECON 560 Class Notes GMM Geeralzed Method of Momets (GMM) I beg by outlg the classcal method of momets techque (Fsher, 95) ad the proceed to geeralzed method of momets (Hase, 98).. radtoal Method of Momets
More information{ }{ ( )} (, ) = ( ) ( ) ( ) Chapter 14 Exercises in Sampling Theory. Exercise 1 (Simple random sampling): Solution:
Chapter 4 Exercses Samplg Theory Exercse (Smple radom samplg: Let there be two correlated radom varables X ad A sample of sze s draw from a populato by smple radom samplg wthout replacemet The observed
More informationMaps on Triangular Matrix Algebras
Maps o ragular Matrx lgebras HMED RMZI SOUROUR Departmet of Mathematcs ad Statstcs Uversty of Vctora Vctora, BC V8W 3P4 CND sourour@mathuvcca bstract We surveys results about somorphsms, Jorda somorphsms,
More informationLine Fitting and Regression
Marquette Uverst MSCS6 Le Fttg ad Regresso Dael B. Rowe, Ph.D. Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Coprght 8 b Marquette Uverst Least Squares Regresso MSCS6 For LSR we have pots
More informationLattices. Mathematical background
Lattces Mathematcal backgroud Lattces : -dmesoal Eucldea space. That s, { T x } x x = (,, ) :,. T T If x= ( x,, x), y = ( y,, y), the xy, = xy (er product of xad y) x = /2 xx, (Eucldea legth or orm of
More informationA Penalty Function Algorithm with Objective Parameters and Constraint Penalty Parameter for Multi-Objective Programming
Aerca Joural of Operatos Research, 4, 4, 33-339 Publshed Ole Noveber 4 ScRes http://wwwscrporg/oural/aor http://ddoorg/436/aor4463 A Pealty Fucto Algorth wth Obectve Paraeters ad Costrat Pealty Paraeter
More informationAhmed Elgamal. MDOF Systems & Modal Analysis
DOF Systems & odal Aalyss odal Aalyss (hese otes cover sectos from Ch. 0, Dyamcs of Structures, Al Chopra, Pretce Hall, 995). Refereces Dyamcs of Structures, Al K. Chopra, Pretce Hall, New Jersey, ISBN
More informationROOT-LOCUS ANALYSIS. Lecture 11: Root Locus Plot. Consider a general feedback control system with a variable gain K. Y ( s ) ( ) K
ROOT-LOCUS ANALYSIS Coder a geeral feedback cotrol yte wth a varable ga. R( Y( G( + H( Root-Locu a plot of the loc of the pole of the cloed-loop trafer fucto whe oe of the yte paraeter ( vared. Root locu
More informationfor each of its columns. A quick calculation will verify that: thus m < dim(v). Then a basis of V with respect to which T has the form: A
Desty of dagoalzable square atrces Studet: Dael Cervoe; Metor: Saravaa Thyagaraa Uversty of Chcago VIGRE REU, Suer 7. For ths etre aer, we wll refer to V as a vector sace over ad L(V) as the set of lear
More informationX X X E[ ] E X E X. is the ()m n where the ( i,)th. j element is the mean of the ( i,)th., then
Secto 5 Vectors of Radom Varables Whe workg wth several radom varables,,..., to arrage them vector form x, t s ofte coveet We ca the make use of matrx algebra to help us orgaze ad mapulate large umbers
More informationEcon 388 R. Butler 2016 rev Lecture 5 Multivariate 2 I. Partitioned Regression and Partial Regression Table 1: Projections everywhere
Eco 388 R. Butler 06 rev Lecture 5 Multvarate I. Parttoed Regresso ad Partal Regresso Table : Projectos everywhere P = ( ) ad M = I ( ) ad s a vector of oes assocated wth the costat term Sample Model Regresso
More informationChapter 2 Supplemental Text Material
-. Models for the Data ad the t-test Chapter upplemetal Text Materal The model preseted the text, equato (-3) s more properl called a meas model. ce the mea s a locato parameter, ths tpe of model s also
More informationIntegral Equation Methods. Jacob White. Thanks to Deepak Ramaswamy, Michal Rewienski, Xin Wang and Karen Veroy
Itroducto to Smulato - Lecture 22 Itegral Equato ethods Jacob Whte Thaks to Deepak Ramaswamy, chal Rewesk, X Wag ad Kare Veroy Outle Itegral Equato ethods Exteror versus teror problems Start wth usg pot
More informationx y exp λ'. x exp λ 2. x exp 1.
egecosmcd Egevalue-egevector of the secod dervatve operator d /d hs leads to Fourer seres (se, cose, Legedre, Bessel, Chebyshev, etc hs s a eample of a systematc way of geeratg a set of mutually orthogoal
More informationClass 13,14 June 17, 19, 2015
Class 3,4 Jue 7, 9, 05 Pla for Class3,4:. Samplg dstrbuto of sample mea. The Cetral Lmt Theorem (CLT). Cofdece terval for ukow mea.. Samplg Dstrbuto for Sample mea. Methods used are based o CLT ( Cetral
More informationRegresso What s a Model? 1. Ofte Descrbe Relatoshp betwee Varables 2. Types - Determstc Models (o radomess) - Probablstc Models (wth radomess) EPI 809/Sprg 2008 9 Determstc Models 1. Hypothesze
More informationChapter 5. Curve fitting
Chapter 5 Curve ttg Assgmet please use ecell Gve the data elow use least squares regresso to t a a straght le a power equato c a saturato-growthrate equato ad d a paraola. Fd the r value ad justy whch
More informationρ < 1 be five real numbers. The
Lecture o BST 63: Statstcal Theory I Ku Zhag, /0/006 Revew for the prevous lecture Deftos: covarace, correlato Examples: How to calculate covarace ad correlato Theorems: propertes of correlato ad covarace
More information