Sparse Functional Regression
|
|
- Lisa Beasley
- 5 years ago
- Views:
Transcription
1 Sparse Functional Regression Junier B. Olia, Barnabás Póczos, Aarti Singh, Jeff Schneider, Timothy Verstynen Machine Learning Department Robotics Institute Psychology Department Carnegie Mellon Uniersity Pittsburgh, PA 53 Introduction There are a multitude of applications and domains where the study of a mapping that takes in a functional input and outputs a real-alue is of interest. That is, if I is some class of input functions with domain R and range R, then one may be interested in a mapping h : I 7! R: h(f) =Y (Figure (a)). Examples include: a mapping that takes in the time-series of a commodity s price in the past (f is a function with the domain of time and range of price) and outputs the expected price of the commodity in the nearby future; also, a mapping that takes a patient s cardiac monitor s time-series and outputs a health index. Recently, work by [5] has explored this type of regression problem when the input function is a distribution. Furthermore, the general case of an arbitrary functional input is related to functional analysis []. Howeer, it is often expected that the response one is interested in regressing is dependent on not ust one, but many functions. That is, it may be fruitful to consider a mapping h : I... I p 7! R: h(f,...,f p )=Y (Figure (b)). For instance, this is likely the case in regressing the price of a commodity in the future, since the commodity s future price is not only dependent on the history of it own price, but also the history of other commodities prices as well. A response s dependence on multiple functional coariates is especially common in neurological data, where thousands of oxels in the brain may each contain a corresponding function. In fact, in such domains it is not uncommon to hae a number of input functional coariates that far exceeds the number of training instances one has in a data-set. Thus, it would be beneficial to hae an estimator that is sparse in the number of functional coariates used to regress the response against. That is, find an estimate, ĥ, that depends on a small subset {i,...,i S } {,...,p}, such that ĥ(f,...,f p )=ĥs(f i,...,f is ) (Figure (c)). Here we present a semi-parametric estimator to perform sparse regression with multiple input functional coariates and a real-alued response, FuSSO: Functional Shrinkage and Selection Operator. No parametric assumptions are made on the nature of input functions. We shall assume that the response is the result of a sparse set of linear combinations of input functions and other non-paramteric functions {g i }: Y = P hf,g i+. The resulting method is a LASSO-like [7] estimator that effectiely zeros out entire functions from consideration in regressing the response. The estimator was found to be effectie in regressing the age of a subect when gien orientation distribution function (ODF) data for the subect s white matter. Related Work As preiously mentioned, recently [5] explored regression with a mapping that takes in a probability density function and outputs a real alue. Furthermore, [4] studies the case when both the input and outputs are distributions. In addition, functional analysis relates to the study of functional data []. In all of these works, the mappings studied take in only one functional coariate. Howeer, it is not immediately eident how to expand on these ideas to deelop an estimator that simultaneously performs regression and feature selection with multiple function coariates.
2 f f f f f i Y f i Y f p- f p- f p f p (a) Single Functional Coariate (b) Multiple Functional Coariates (c) Sparse Model Figure : (a) Model where mapping takes in a function f and produces a real Y. (b) Model where response Y is dependent on multiple input functions f,...,f p. (c) Sparse model where response Y is dependent on a sparse subset of input functions f,...,f p. To our knowledge, there has been no prior work in studying sparse mappings that take multiple functional inputs and produce a real-alued output. LASSO-like regression estimators that work with functional data include the following. In [3], one has a functional output and seeral realalued coariates. Here, the estimator finds a sparse set of functions to scale by the real alued coariates to produce a functional response. Also, [, ] study the case when one has one functional coariate f and one real alued response that is linearly dependent on f and some function g: Y = hf,gi = R fg. First, in [] the estimator searches for sparsity across waelet basis proection coefficients. In [], sparsity is achieed in the time (input) domain of the d th deriatie of g; i.e. [D d g](t) =for many alues of t where D d is the differential operator. Hence, roughly speaking, [, ] look for sparsity across frequency and time domains respectiely, for the regessing function g. Howeer, these methods do not consider the case where one has many input functional coariates {f,...,f p }, and needs to choose amongst them. That is, [, ] do not proide a method to select among function coariates in an analogous fashion to how the LASSO selects among real-alued coariates. Lastly, it is worth noting that in our estimator we will hae an additie linear model, P hf,g i where we search for {g i } in a broad, non-parametric family such that many g are the zero function. Such P a task is similar in nature to the SpAM estimator [6], in which one also has an additie model g (X ) (in the dimensions of a real ector X) and searches for {g i } in a broad, non-parametric family such that many g are the zero function. Note though, that in the SpAM model, the {g i } functions are applied to real coariates ia a function ealuation. In the FuSSO model, {g i } are applied to functional coariates ia an inner product; that is, FuSSO works oer functional, not real-alued coariates, unlike SpAM. 3 Model In order to better understand FuSSO s model we draw seeral analogies to real-alued linear regression and Group-LASSO [9]. First, consider a model for typical real-alued linear regression with a data-set of input-output pairs {(X i,y i )} N i= : Y i = hx i,wi + i, where Y i R, X i R d,w R d, i iid N (, dx ), and hx i,wi = X i w. If, instead, one were working with functional data {(f (i),y i )} N i=, where f (i) :[, ] 7! R and f (i) L [, ], one might similarly consider a linear model: Y i = hf (i),gi + i, where g :[, ] 7! R, and hf (i),gi = Z f (i) (t)g(t)dt.
3 If = {' m } is an orthonormal basis for L [, ] [8] then we hae that Z f (i) (x) = m (i) ' m (x), where m (i) = f (i) (t)' m (t)dt. () Similarly, g(x) = P m' m (x). Thus, Y i = hf (i),gi + i = h m (i) ' m (x), = m m + i, k= where the last step follows from orthonormality of. k' k (x)i + i = k= m kh' m (x), ' k (x)i + i Going back to the real-alued coariate case, if instead of haing one feature ector per data instance, X i R d, one had p feature ectors associated with each data instance: {X i apple apple p, X i R d }, an additie linear model could be used for regression: Y i = hx id,w d i + i, where w,...,w d R d. d= Similarly, in the functional case, one may hae p functions associated with data instance i: {f (i) apple apple p, f (i) L [, ]}. Then, an additie linear model would be: Y i = hf (i),g i + i = + i, () where g,...,g p L [, ], and and are proection coefficients. Suppose that one has few obserations relatie to the number of features (N p). In the realalued case, in order to effectiely find a solution for w =(w T,...,wp T ) T one may search for a group sparse solution where many w =. To do so, one may consider the following Group-LASSO regression: w? = argmin w N ky X w k + N kw k, (3) where X is the N d matrix X =[X...X N ] T, Y =(Y,...,Y N ) T, and k k is the Euclidean norm. If in the functional case () one also has that N p, one may set up a similar optimization to (3), whose direct analogue is: g? = i hf (i),g ia + N kg k; (4) g N equialently,? = argmin N where g? = {g? i }p i= = {P i i=? im ' m, } p i=. A u X + N t, (5) Howeer, it is intractable to assume that one is able to directly obsere functional inputs {f (i) apple i apple N, apple apple p}. Thus, we shall instead assume that one obseres {~y (i) apple i apple N, apple apple p} where ~y (i) = ~ f (i) + (i), f ~ (i) = f (i) T (/n), f (i) (/n),...,f (i) (), (i) iid N (, I). (6) 3
4 That is, we obsere a grid of n noisy alues for each functional input. Then, one may estimate as: = n ~' T m~y (i) = n ~' T m( f ~ (i) + (i) )= + (i) where ~' m =(' m (/n), ' m (/n),...,' m ()) T. Furthermore, we may truncate the number of basis functions used to express f (i) to M n, estimating it as: f (i) (x) = Using the truncated estimate (7), one has: (i) h f (x),g i = XM n XM n Hence, using the approximations (7), (5) becomes: ˆ XM n = i N i= = argmin N ky ' m(x). (7), (i) and k f (x)k = A u X t Mn ( ). u X + t Mn N (8) à k + N k k, (9) where à is the N M n matrix with alues Ã(i, m) = and =(,..., M n ) T. Note that one need not consider proection coefficients for m>m n since such proection coefficients will not decrease the MSE term in (8) (because =for m>m n), and 6=for m>m n increases the norm penalty term in (8). Hence, we see that our sparse functional estimates are a Group-LASSO problem on the proection coefficients. In a future publication, we shall show that if {f (i) }, and {g } are in a Sobole function class and some other mild assumptions hold, then our estimator is asymptotically sparsistent. 4 Experiments We tested the FuSSO estimator with neurological data. It consisted of 89 total subects. Orientation distribution function (ODF) (Figure (a)) data was proided for each subect in a template space for white-matter oxels; a total of oer 5 thousand oxel s ODFs were regressed on. We looked to regress a subect s age gien his/her respectie ODF data. The proection coefficients for the ODFs at each oxel were estimated using the cosine basis. The FuSSO estimator gae a held out MSE of 7.855, where the ariance for age was Frequency Frequency 5 (a) Example ODF Age (b) Ages (c) Actie Voxels 3 Absolute Error (d) Errors Figure : (a) An example ODF for a oxel. (b) Histogram of ages for subects. (c) Voxels in the support of model shown in blue. (d) Histogram of held out error magnitudes. 4
5 References [] F. Ferraty and P. Vieu. Nonparametric functional data analysis: theory and practice. Springer, 6. [] Gareth M James, Jing Wang, and Ji Zhu. Functional linear regression that s interpretable. The Annals of Statistics, pages 83 8, 9. [3] Nicola Mingotti, Rosa E Lillo, and Juan Romo. Lasso ariable selection in functional regression. 3. [4] Junier B Olia, Barnabás Póczos, and Jeff Schneider. Distribution to distribution regression. [5] B. Poczos, A. Rinaldo, A. Singh, and L Wasserman. Distribution-Free Distribution Regression. arxi preprint arxi:3.8,. [6] Pradeep Raikumar, John Lafferty, Han Liu, and Larry Wasserman. Sparse additie models. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 7(5):9 3, 9. [7] Robert Tibshirani. Regression shrinkage and selection ia the lasso. Journal of the Royal Statistical Society. Series B (Methodological), pages 67 88, 996. [8] Alexandre B Tsybako. Introduction to nonparametric estimation. Springer, 8. [9] Ming Yuan and Yi Lin. Model selection and estimation in regression with grouped ariables. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 68():49 67, 6. [] Yihong Zhao, R Todd Ogden, and Philip T Reiss. Waelet-based lasso in functional linear regression. Journal of Computational and Graphical Statistics, (3):6 67,. 5
FuSSO: Functional Shrinkage and Selection Operator
Junier B. Oliva Barnabás Póczos Timothy Verstynen Aarti Singh Jeff Schneider Fang-Cheng eh Wen-ih Tseng Carnegie Mellon University ational Taiwan University Abstract We present the FuSSO, a functional
More informationModeling Highway Traffic Volumes
Modeling Highway Traffic Volumes Tomáš Šingliar1 and Miloš Hauskrecht 1 Computer Science Dept, Uniersity of Pittsburgh, Pittsburgh, PA 15260 {tomas, milos}@cs.pitt.edu Abstract. Most traffic management
More informationApplied Machine Learning for Design Optimization in Cosmology, Neuroscience, and Drug Discovery
Applied Machine Learning for Design Optimization in Cosmology, Neuroscience, and Drug Discovery Barnabas Poczos Machine Learning Department Carnegie Mellon University Machine Learning Technologies and
More informationDistribution-Free Distribution Regression
Distribution-Free Distribution Regression Barnabás Póczos, Alessandro Rinaldo, Aarti Singh and Larry Wasserman AISTATS 2013 Presented by Esther Salazar Duke University February 28, 2014 E. Salazar (Reading
More informationSUPPLEMENTARY MATERIAL. Authors: Alan A. Stocker (1) and Eero P. Simoncelli (2)
SUPPLEMENTARY MATERIAL Authors: Alan A. Stocker () and Eero P. Simoncelli () Affiliations: () Dept. of Psychology, Uniersity of Pennsylania 34 Walnut Street 33C Philadelphia, PA 94-68 U.S.A. () Howard
More informationAsymptotic Normality of an Entropy Estimator with Exponentially Decaying Bias
Asymptotic Normality of an Entropy Estimator with Exponentially Decaying Bias Zhiyi Zhang Department of Mathematics and Statistics Uniersity of North Carolina at Charlotte Charlotte, NC 28223 Abstract
More informationarxiv: v1 [stat.ml] 15 Feb 2018
1 : A New Algorithm for Streaming PCA arxi:1802.054471 [stat.ml] 15 Feb 2018 Puyudi Yang, Cho-Jui Hsieh, Jane-Ling Wang Uniersity of California, Dais pydyang, chohsieh, janelwang@ucdais.edu Abstract In
More informationcxx ab.ec Warm up OH 2 ax 16 0 axtb Fix any a, b, c > What is the x 2 R that minimizes ax 2 + bx + c
Warm up D cai.yo.ie p IExrL9CxsYD Sglx.Ddl f E Luo fhlexi.si dbll Fix any a, b, c > 0. 1. What is the x 2 R that minimizes ax 2 + bx + c x a b Ta OH 2 ax 16 0 x 1 Za fhkxiiso3ii draulx.h dp.d 2. What is
More informationAssignment 4 (Solutions) NPTEL MOOC (Bayesian/ MMSE Estimation for MIMO/OFDM Wireless Communications)
Assignment 4 Solutions NPTEL MOOC Bayesian/ MMSE Estimation for MIMO/OFDM Wireless Communications The system model can be written as, y hx + The MSE of the MMSE estimate ĥ of the aboe mentioned system
More informationAn Optimal Split-Plot Design for Performing a Mixture-Process Experiment
Science Journal of Applied Mathematics and Statistics 217; 5(1): 15-23 http://www.sciencepublishinggroup.com/j/sjams doi: 1.11648/j.sjams.21751.13 ISSN: 2376-9491 (Print); ISSN: 2376-9513 (Online) An Optimal
More informationBalanced Partitions of Vector Sequences
Balanced Partitions of Vector Sequences Imre Bárány Benjamin Doerr December 20, 2004 Abstract Let d,r N and be any norm on R d. Let B denote the unit ball with respect to this norm. We show that any sequence
More informationA matrix Method for Interval Hermite Curve Segmentation O. Ismail, Senior Member, IEEE
International Journal of Video&Image Processing Network Security IJVIPNS-IJENS Vol:15 No:03 7 A matrix Method for Interal Hermite Cure Segmentation O. Ismail, Senior Member, IEEE Abstract Since the use
More informationLearning discrete graphical models via generalized inverse covariance matrices
Learning discrete graphical models via generalized inverse covariance matrices Duzhe Wang, Yiming Lv, Yongjoon Kim, Young Lee Department of Statistics University of Wisconsin-Madison {dwang282, lv23, ykim676,
More informationLearning Markov Network Structure using Brownian Distance Covariance
arxiv:.v [stat.ml] Jun 0 Learning Markov Network Structure using Brownian Distance Covariance Ehsan Khoshgnauz May, 0 Abstract In this paper, we present a simple non-parametric method for learning the
More informationBAGUS: Bayesian Regularization for Graphical Models with Unequal Shrinkage
BAGUS: Bayesian Regularization for Graphical Models with Unequal Shrinkage Lingrui Gan, Naveen N. Narisetty, Feng Liang Department of Statistics University of Illinois at Urbana-Champaign Problem Statement
More informationOBSERVATIONS ON BAGGING
OBSERVATIONS ON BAGGING Andreas Buja and Werner Stuetzle Uniersity of Pennsylania and Uniersity of Washington Abstract: Bagging is a deice intended for reducing the prediction error of learning algorithms.
More informationAstrometric Errors Correlated Strongly Across Multiple SIRTF Images
Astrometric Errors Correlated Strongly Across Multiple SIRTF Images John Fowler 28 March 23 The possibility exists that after pointing transfer has been performed for each BCD (i.e. a calibrated image
More informationA Regularization Framework for Learning from Graph Data
A Regularization Framework for Learning from Graph Data Dengyong Zhou Max Planck Institute for Biological Cybernetics Spemannstr. 38, 7076 Tuebingen, Germany Bernhard Schölkopf Max Planck Institute for
More informationdifferent formulas, depending on whether or not the vector is in two dimensions or three dimensions.
ectors The word ector comes from the Latin word ectus which means carried. It is best to think of a ector as the displacement from an initial point P to a terminal point Q. Such a ector is expressed as
More informationStability Approach to Regularization Selection (StARS) for High Dimensional Graphical Models
Stability Approach to Regularization Selection (StARS) for High Dimensional Graphical Models Han Liu Kathryn Roeder Larry Wasserman Carnegie Mellon University Pittsburgh, PA 15213 Abstract A challenging
More informationGeometric ergodicity of the Bayesian lasso
Geometric ergodicity of the Bayesian lasso Kshiti Khare and James P. Hobert Department of Statistics University of Florida June 3 Abstract Consider the standard linear model y = X +, where the components
More informationSparse Nonparametric Density Estimation in High Dimensions Using the Rodeo
Sparse Nonparametric Density Estimation in High Dimensions Using the Rodeo Han Liu John Lafferty Larry Wasserman Statistics Department Computer Science Department Machine Learning Department Carnegie Mellon
More informationThe lasso. Patrick Breheny. February 15. The lasso Convex optimization Soft thresholding
Patrick Breheny February 15 Patrick Breheny High-Dimensional Data Analysis (BIOS 7600) 1/24 Introduction Last week, we introduced penalized regression and discussed ridge regression, in which the penalty
More informationMath 144 Activity #9 Introduction to Vectors
144 p 1 Math 144 ctiity #9 Introduction to Vectors Often times you hear people use the words speed and elocity. Is there a difference between the two? If so, what is the difference? Discuss this with your
More informationEstimation of Efficiency with the Stochastic Frontier Cost. Function and Heteroscedasticity: A Monte Carlo Study
Estimation of Efficiency ith the Stochastic Frontier Cost Function and Heteroscedasticity: A Monte Carlo Study By Taeyoon Kim Graduate Student Oklahoma State Uniersity Department of Agricultural Economics
More informationLECTURE 3 3.1Rules of Vector Differentiation
LETURE 3 3.1Rules of Vector Differentiation We hae defined three kinds of deriaties inoling the operator grad( ) i j k, x y z 1 3 di(., x y z curl( i x 1 j y k z 3 d The good news is that you can apply
More information0 a 3 a 2 a 3 0 a 1 a 2 a 1 0
Chapter Flow kinematics Vector and tensor formulae This introductory section presents a brief account of different definitions of ector and tensor analysis that will be used in the following chapters.
More informationVariance Reduction for Stochastic Gradient Optimization
Variance Reduction for Stochastic Gradient Optimization Chong Wang Xi Chen Alex Smola Eric P. Xing Carnegie Mellon Uniersity, Uniersity of California, Berkeley {chongw,xichen,epxing}@cs.cmu.edu alex@smola.org
More informationPrediction of anode arc root position in a DC arc plasma torch
Prediction of anode arc root position in a DC arc plasma torch He-Ping Li 1, E. Pfender 1, Xi Chen 1 Department of Mechanical Engineering, Uniersity of Minnesota, Minneapolis, MN 55455, USA Department
More information6.1.1 Angle between Two Lines Intersection of Two lines Shortest Distance from a Point to a Line
CHAPTER 6 : VECTORS 6. Lines in Space 6.. Angle between Two Lines 6.. Intersection of Two lines 6..3 Shortest Distance from a Point to a Line 6. Planes in Space 6.. Intersection of Two Planes 6.. Angle
More informationResidual migration in VTI media using anisotropy continuation
Stanford Exploration Project, Report SERGEY, Noember 9, 2000, pages 671?? Residual migration in VTI media using anisotropy continuation Tariq Alkhalifah Sergey Fomel 1 ABSTRACT We introduce anisotropy
More information4. A Physical Model for an Electron with Angular Momentum. An Electron in a Bohr Orbit. The Quantum Magnet Resulting from Orbital Motion.
4. A Physical Model for an Electron with Angular Momentum. An Electron in a Bohr Orbit. The Quantum Magnet Resulting from Orbital Motion. We now hae deeloped a ector model that allows the ready isualization
More informationDATA MINING AND MACHINE LEARNING
DATA MINING AND MACHINE LEARNING Lecture 5: Regularization and loss functions Lecturer: Simone Scardapane Academic Year 2016/2017 Table of contents Loss functions Loss functions for regression problems
More informationReview of Matrices and Vectors 1/45
Reiew of Matrices and Vectors /45 /45 Definition of Vector: A collection of comple or real numbers, generally put in a column [ ] T "! Transpose + + + b a b a b b a a " " " b a b a Definition of Vector
More informationOn general error distributions
ProbStat Forum, Volume 6, October 3, Pages 89 95 ISSN 974-335 ProbStat Forum is an e-journal. For details please isit www.probstat.org.in On general error distributions R. Vasudea, J. Vasantha Kumari Department
More informationEfficient solution of interval optimization problem
Math Meth Oper Res (212) 76:273 288 DOI 1.17/s186-12-399- ORIGINAL ARTICLE Efficient solution of interal optimization problem A. K. Bhurjee G. Panda Receied: 6 June 212 / Accepted: 4 August 212 / Published
More informationarxiv: v1 [physics.comp-ph] 17 Jan 2014
An efficient method for soling a correlated multi-item inentory system Chang-Yong Lee and Dongu Lee The Department of Industrial & Systems Engineering, Kongu National Uniersity, Kongu 34-70 South Korea
More informationMachine Learning for Big Data CSE547/STAT548, University of Washington Emily Fox February 4 th, Emily Fox 2014
Case Study 3: fmri Prediction Fused LASSO LARS Parallel LASSO Solvers Machine Learning for Big Data CSE547/STAT548, University of Washington Emily Fox February 4 th, 2014 Emily Fox 2014 1 LASSO Regression
More informationTowards Green Distributed Storage Systems
Towards Green Distributed Storage Systems Abdelrahman M. Ibrahim, Ahmed A. Zewail, and Aylin Yener Wireless Communications and Networking Laboratory (WCAN) Electrical Engineering Department The Pennsylania
More informationSemiparametric Analysis of Heterogeneous Data Using Varying-Scale Generalized Linear Models
Semiparametric Analysis of Heterogeneous Data Using Varying-Scale Generalized Linear Models Minge XIE, Douglas G. SIMPSON, and Raymond J. CARROLL This article describes a class of heteroscedastic generalized
More informationIs the test error unbiased for these programs? 2017 Kevin Jamieson
Is the test error unbiased for these programs? 2017 Kevin Jamieson 1 Is the test error unbiased for this program? 2017 Kevin Jamieson 2 Simple Variable Selection LASSO: Sparse Regression Machine Learning
More informationSTATISTICAL MACHINE LEARNING FOR STRUCTURED AND HIGH DIMENSIONAL DATA
AFRL-OSR-VA-TR-2014-0234 STATISTICAL MACHINE LEARNING FOR STRUCTURED AND HIGH DIMENSIONAL DATA Larry Wasserman CARNEGIE MELLON UNIVERSITY 0 Final Report DISTRIBUTION A: Distribution approved for public
More informationChris Fraley and Daniel Percival. August 22, 2008, revised May 14, 2010
Model-Averaged l 1 Regularization using Markov Chain Monte Carlo Model Composition Technical Report No. 541 Department of Statistics, University of Washington Chris Fraley and Daniel Percival August 22,
More informationLearning the Semantic Correlation: An Alternative Way to Gain from Unlabeled Text
Learning the Semantic Correlation: An Alternative Way to Gain from Unlabeled Text Yi Zhang Machine Learning Department Carnegie Mellon University yizhang1@cs.cmu.edu Jeff Schneider The Robotics Institute
More informationECE G: Special Topics in Signal Processing: Sparsity, Structure, and Inference
ECE 18-898G: Special Topics in Signal Processing: Sparsity, Structure, and Inference Sparse Recovery using L1 minimization - algorithms Yuejie Chi Department of Electrical and Computer Engineering Spring
More informationTHE FIFTH DIMENSION EQUATIONS
JP Journal of Mathematical Sciences Volume 7 Issues 1 & 013 Pages 41-46 013 Ishaan Publishing House This paper is aailable online at http://www.iphsci.com THE FIFTH DIMENSION EQUATIONS Niittytie 1B16 03100
More informationNoise constrained least mean absolute third algorithm
Noise constrained least mean absolute third algorithm Sihai GUAN 1 Zhi LI 1 Abstract: he learning speed of an adaptie algorithm can be improed by properly constraining the cost function of the adaptie
More informationOn computing Gaussian curvature of some well known distribution
Theoretical Mathematics & Applications, ol.3, no.4, 03, 85-04 ISSN: 79-9687 (print), 79-9709 (online) Scienpress Ltd, 03 On computing Gaussian curature of some well known distribution William W.S. Chen
More informationUNDERSTAND MOTION IN ONE AND TWO DIMENSIONS
SUBAREA I. COMPETENCY 1.0 UNDERSTAND MOTION IN ONE AND TWO DIMENSIONS MECHANICS Skill 1.1 Calculating displacement, aerage elocity, instantaneous elocity, and acceleration in a gien frame of reference
More informationv v Downloaded 01/11/16 to Redistribution subject to SEG license or copyright; see Terms of Use at
The pseudo-analytical method: application of pseudo-laplacians to acoustic and acoustic anisotropic wae propagation John T. Etgen* and Serre Brandsberg-Dahl Summary We generalize the pseudo-spectral method
More informationOnline Dictionary Learning with Group Structure Inducing Norms
Online Dictionary Learning with Group Structure Inducing Norms Zoltán Szabó 1, Barnabás Póczos 2, András Lőrincz 1 1 Eötvös Loránd University, Budapest, Hungary 2 Carnegie Mellon University, Pittsburgh,
More informationNon-Surjective Finite Alphabet Iterative Decoders
IEEE ICC 2016 - Communications Theory Non-Surjectie Finite Alphabet Iteratie Decoders Thien Truong Nguyen-Ly, Khoa Le,V.Sain, D. Declercq, F. Ghaffari and O. Boncalo CEA-LETI, MINATEC Campus, Grenoble,
More informationDoubly Decomposing Nonparametric Tensor Regression (ICML 2016)
Doubly Decomposing Nonparametric Tensor Regression (ICML 2016) M.Imaizumi (Univ. of Tokyo / JSPS DC) K.Hayashi (AIST / JST ERATO) 2016/08/10 Outline Topic Nonparametric Regression with Tensor input Model
More informationPatterns of Non-Simple Continued Fractions
Patterns of Non-Simple Continued Fractions Jesse Schmieg A final report written for the Uniersity of Minnesota Undergraduate Research Opportunities Program Adisor: Professor John Greene March 01 Contents
More informationNotes on Linear Minimum Mean Square Error Estimators
Notes on Linear Minimum Mean Square Error Estimators Ça gatay Candan January, 0 Abstract Some connections between linear minimum mean square error estimators, maximum output SNR filters and the least square
More informationIs the test error unbiased for these programs?
Is the test error unbiased for these programs? Xtrain avg N o Preprocessing by de meaning using whole TEST set 2017 Kevin Jamieson 1 Is the test error unbiased for this program? e Stott see non for f x
More informationUnderstanding the relationship between Functional and Structural Connectivity of Brain Networks
Understanding the relationship between Functional and Structural Connectivity of Brain Networks Sashank J. Reddi Machine Learning Department, Carnegie Mellon University SJAKKAMR@CS.CMU.EDU Abstract Background.
More informationLESSON 4: INTEGRATION BY PARTS (I) MATH FALL 2018
LESSON 4: INTEGRATION BY PARTS (I) MATH 6 FALL 8 ELLEN WELD. Integration by Parts We introduce another method for ealuating integrals called integration by parts. The key is the following : () u d = u
More informationAn Improved 1-norm SVM for Simultaneous Classification and Variable Selection
An Improved 1-norm SVM for Simultaneous Classification and Variable Selection Hui Zou School of Statistics University of Minnesota Minneapolis, MN 55455 hzou@stat.umn.edu Abstract We propose a novel extension
More informationFast Algorithms for LAD Lasso Problems
Fast Algorithms for LAD Lasso Problems Robert J. Vanderbei 2015 Nov 3 http://www.princeton.edu/ rvdb INFORMS Philadelphia Lasso Regression The problem is to solve a sparsity-encouraging regularized regression
More information6. Regularized linear regression
Foundations of Machine Learning École Centrale Paris Fall 2015 6. Regularized linear regression Chloé-Agathe Azencot Centre for Computational Biology, Mines ParisTech chloe agathe.azencott@mines paristech.fr
More informationOPTIMAL RESOLVABLE DESIGNS WITH MINIMUM PV ABERRATION
Statistica Sinica 0 (010), 715-73 OPTIMAL RESOLVABLE DESIGNS WITH MINIMUM PV ABERRATION J. P. Morgan Virginia Tech Abstract: Amongst resolable incomplete block designs, affine resolable designs are optimal
More informationLecture 21: Physical Brownian Motion II
Lecture 21: Physical Brownian Motion II Scribe: Ken Kamrin Department of Mathematics, MIT May 3, 25 Resources An instructie applet illustrating physical Brownian motion can be found at: http://www.phy.ntnu.edu.tw/jaa/gas2d/gas2d.html
More informationSparse regression. Optimization-Based Data Analysis. Carlos Fernandez-Granda
Sparse regression Optimization-Based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_spring16 Carlos Fernandez-Granda 3/28/2016 Regression Least-squares regression Example: Global warming Logistic
More informationAlternatives to Basis Expansions. Kernels in Density Estimation. Kernels and Bandwidth. Idea Behind Kernel Methods
Alternatives to Basis Expansions Basis expansions require either choice of a discrete set of basis or choice of smoothing penalty and smoothing parameter Both of which impose prior beliefs on data. Alternatives
More informationSupport Vector Machine For Functional Data Classification
Support Vector Machine For Functional Data Classification Nathalie Villa 1 and Fabrice Rossi 2 1- Université Toulouse Le Mirail - Equipe GRIMM 5allées A. Machado, 31058 Toulouse cedex 1 - FRANCE 2- Projet
More informationStatistical Learning with the Lasso, spring The Lasso
Statistical Learning with the Lasso, spring 2017 1 Yeast: understanding basic life functions p=11,904 gene values n number of experiments ~ 10 Blomberg et al. 2003, 2010 The Lasso fmri brain scans function
More informationLASSO Review, Fused LASSO, Parallel LASSO Solvers
Case Study 3: fmri Prediction LASSO Review, Fused LASSO, Parallel LASSO Solvers Machine Learning for Big Data CSE547/STAT548, University of Washington Sham Kakade May 3, 2016 Sham Kakade 2016 1 Variable
More informationCases of integrability corresponding to the motion of a pendulum on the two-dimensional plane
Cases of integrability corresponding to the motion of a pendulum on the two-dimensional plane MAXIM V. SHAMOLIN Lomonoso Moscow State Uniersity Institute of Mechanics Michurinskii Ae.,, 99 Moscow RUSSIAN
More informationA note on the group lasso and a sparse group lasso
A note on the group lasso and a sparse group lasso arxiv:1001.0736v1 [math.st] 5 Jan 2010 Jerome Friedman Trevor Hastie and Robert Tibshirani January 5, 2010 Abstract We consider the group lasso penalty
More informationDEVIL PHYSICS THE BADDEST CLASS ON CAMPUS AP PHYSICS
DEVIL PHYSICS THE BADDEST CLASS ON CAMPUS AP PHYSICS LSN 3-7: PROJECTILE MOTION IS PARABOLIC LSN 3-8: RELATIVE VELOCITY Questions From Reading Actiity? Big Idea(s): The interactions of an object with other
More informationSparse Nonparametric Density Estimation in High Dimensions Using the Rodeo
Outline in High Dimensions Using the Rodeo Han Liu 1,2 John Lafferty 2,3 Larry Wasserman 1,2 1 Statistics Department, 2 Machine Learning Department, 3 Computer Science Department, Carnegie Mellon University
More informationEquivalence of Multi-Formulated Optimal Slip Control for Vehicular Anti-Lock Braking System
Preprints of the 19th World Congress The International Federation of Automatic Control Cape Town, South Africa. August 24-29, 214 Equialence of Multi-Formulated Optimal Slip Control for Vehicular Anti-Lock
More informationStability Approach to Regularization Selection (StARS) for High Dimensional Graphical Models
Stability Approach to Regularization Selection (StARS) for High Dimensional Graphical Models arxiv:1006.3316v1 [stat.ml] 16 Jun 2010 Contents Han Liu, Kathryn Roeder and Larry Wasserman Carnegie Mellon
More informationConvex envelopes, cardinality constrained optimization and LASSO. An application in supervised learning: support vector machines (SVMs)
ORF 523 Lecture 8 Princeton University Instructor: A.A. Ahmadi Scribe: G. Hall Any typos should be emailed to a a a@princeton.edu. 1 Outline Convexity-preserving operations Convex envelopes, cardinality
More informationInsights into Cross-validation
Noname manuscript No. (will be inserted by the editor) Insights into Cross-alidation Amit Dhurandhar Alin Dobra Receied: date / Accepted: date Abstract Cross-alidation is one of the most widely used techniques,
More informationOnline Companion to Pricing Services Subject to Congestion: Charge Per-Use Fees or Sell Subscriptions?
Online Companion to Pricing Serices Subject to Congestion: Charge Per-Use Fees or Sell Subscriptions? Gérard P. Cachon Pnina Feldman Operations and Information Management, The Wharton School, Uniersity
More informationAn Alternative Characterization of Hidden Regular Variation in Joint Tail Modeling
An Alternatie Characterization of Hidden Regular Variation in Joint Tail Modeling Grant B. Weller Daniel S. Cooley Department of Statistics, Colorado State Uniersity, Fort Collins, CO USA May 22, 212 Abstract
More informationStatistical Inference
Statistical Inference Liu Yang Florida State University October 27, 2016 Liu Yang, Libo Wang (Florida State University) Statistical Inference October 27, 2016 1 / 27 Outline The Bayesian Lasso Trevor Park
More informationChapter 1: Kinematics of Particles
Chapter 1: Kinematics of Particles 1.1 INTRODUCTION Mechanics the state of rest of motion of bodies subjected to the action of forces Static equilibrium of a body that is either at rest or moes with constant
More informationState-space Modelling of Hysteresis-based Control Schemes
European Control Conference (ECC) July 7-9,, Zürich, Switzerland. State-space Modelling of Hysteresis-based Control Schemes Soumya Kundu Ian A. Hiskens Abstract The paper deelops a state-space model for
More informationThe Nonparanormal skeptic
The Nonpara skeptic Han Liu Johns Hopkins University, 615 N. Wolfe Street, Baltimore, MD 21205 USA Fang Han Johns Hopkins University, 615 N. Wolfe Street, Baltimore, MD 21205 USA Ming Yuan Georgia Institute
More informationLast Lecture Recap. UVA CS / Introduc8on to Machine Learning and Data Mining. Lecture 6: Regression Models with Regulariza8on
UVA CS 45 - / 65 7 Introduc8on to Machine Learning and Data Mining Lecture 6: Regression Models with Regulariza8on Yanun Qi / Jane University of Virginia Department of Computer Science Last Lecture Recap
More informationSPECTRAL analysis of time-series recorded from the. Robust Estimation of Sparse Narrowband Spectra from Binary Neuronal Spiking Data
Robust Estimation of Sparse arrowband Spectra from Binary euronal Spiing Data Sina Miran, Patric L. Purdon, Emery. Brown, and Behtash Babadi Abstract Objectie: Characterizing the spectral properties of
More informationSparse Additive Functional and kernel CCA
Sparse Additive Functional and kernel CCA Sivaraman Balakrishnan* Kriti Puniyani* John Lafferty *Carnegie Mellon University University of Chicago Presented by Miao Liu 5/3/2013 Canonical correlation analysis
More informationJournal of Computational and Applied Mathematics. New matrix iterative methods for constraint solutions of the matrix
Journal of Computational and Applied Mathematics 35 (010 76 735 Contents lists aailable at ScienceDirect Journal of Computational and Applied Mathematics journal homepage: www.elseier.com/locate/cam New
More informationSELECTION, SIZING, AND OPERATION OF CONTROL VALVES FOR GASES AND LIQUIDS Class # 6110
SELECTION, SIZIN, AND OERATION OF CONTROL VALVES FOR ASES AND LIUIDS Class # 6110 Ross Turbiille Sales Engineer Fisher Controls International Inc. 301 S. First Aenue Marshalltown, Iowa USA Introduction
More informationTales from fmri Learning from limited labeled data. Gae l Varoquaux
Tales from fmri Learning from limited labeled data Gae l Varoquaux fmri data p 100 000 voxels per map Heavily correlated + structured noise Low SNR: 5% 13 db Brain response maps (activation) n Hundreds,
More informationSparse Bayesian Logistic Regression with Hierarchical Prior and Variational Inference
Sparse Bayesian Logistic Regression with Hierarchical Prior and Variational Inference Shunsuke Horii Waseda University s.horii@aoni.waseda.jp Abstract In this paper, we present a hierarchical model which
More informationStatistical Machine Learning for Structured and High Dimensional Data
Statistical Machine Learning for Structured and High Dimensional Data (FA9550-09- 1-0373) PI: Larry Wasserman (CMU) Co- PI: John Lafferty (UChicago and CMU) AFOSR Program Review (Jan 28-31, 2013, Washington,
More information(a) During the first part of the motion, the displacement is x 1 = 40 km and the time interval is t 1 (30 km / h) (80 km) 40 km/h. t. (2.
Chapter 3. Since the trip consists of two parts, let the displacements during first and second parts of the motion be x and x, and the corresponding time interals be t and t, respectiely. Now, because
More informationWeiss-Weinstein Bounds for Various Priors
Weiss-Weinstein Bounds for Various Priors Florian Xaer, Christoph F. Mecklenbräuker, Peter Gerstoft, and Gerald Matz Abstract We address analytic solutions of the Weiss-Weinstein bound (WWB), which lower
More informationSTAT 535 Lecture 5 November, 2018 Brief overview of Model Selection and Regularization c Marina Meilă
STAT 535 Lecture 5 November, 2018 Brief overview of Model Selection and Regularization c Marina Meilă mmp@stat.washington.edu Reading: Murphy: BIC, AIC 8.4.2 (pp 255), SRM 6.5 (pp 204) Hastie, Tibshirani
More informationA New Extended Uniform Distribution
International Journal of Statistical Distriutions and Applications 206; 2(3): 3-4 http://wwwsciencepulishinggroupcom/j/ijsda doi: 0648/jijsd20602032 ISS: 2472-3487 (Print); ISS: 2472-309 (Online) A ew
More informationinforms DOI /moor.xxxx.xxxx c 20xx INFORMS
MATHEMATICS OF OPERATIONS RESEARCH Vol. 00, No. 0, Xxxxxx 20xx, pp. xxx xxx ISSN 0364-765X EISSN 526-547 xx 0000 0xxx informs DOI 0.287/moor.xxxx.xxxx c 20xx INFORMS A Distributional Interpretation of
More informationThe lasso, persistence, and cross-validation
The lasso, persistence, and cross-validation Daniel J. McDonald Department of Statistics Indiana University http://www.stat.cmu.edu/ danielmc Joint work with: Darren Homrighausen Colorado State University
More informationCSC 576: Variants of Sparse Learning
CSC 576: Variants of Sparse Learning Ji Liu Department of Computer Science, University of Rochester October 27, 205 Introduction Our previous note basically suggests using l norm to enforce sparsity in
More informationVariable Selection for Highly Correlated Predictors
Variable Selection for Highly Correlated Predictors Fei Xue and Annie Qu Department of Statistics, University of Illinois at Urbana-Champaign WHOA-PSI, Aug, 2017 St. Louis, Missouri 1 / 30 Background Variable
More informationA possible mechanism to explain wave-particle duality L D HOWE No current affiliation PACS Numbers: r, w, k
A possible mechanism to explain wae-particle duality L D HOWE No current affiliation PACS Numbers: 0.50.-r, 03.65.-w, 05.60.-k Abstract The relationship between light speed energy and the kinetic energy
More informationMotion in Two and Three Dimensions
PH 1-A Fall 014 Motion in Two and Three Dimensions Lectures 4,5 Chapter 4 (Halliday/Resnick/Walker, Fundamentals of Physics 9 th edition) 1 Chapter 4 Motion in Two and Three Dimensions In this chapter
More information