Solving Constrained Lasso and Elastic Net Using
|
|
- Maude Hunt
- 5 years ago
- Views:
Transcription
1 Solving Constrained Lasso and Elasti Net Using ν s Carlos M. Alaı z, Alberto Torres and Jose R. Dorronsoro Universidad Auto noma de Madrid - Departamento de Ingenierı a Informa tia Toma s y Valiente 11, 8049 Madrid - Spain Abstrat. Many important linear sparse models have at its ore the Lasso problem, for whih the algorithm is often onsidered as the urrent state of the art. Reently M. Jaggi has observed that Constrained Lasso (CL) an be redued to a -like problem, whih opens the way to use effiient algorithms to solve CL. We will refine Jaggi s arguments to redue CL as well as onstrained Elasti Net to a Nearest Point Problem and show experimentally that the well known LIB library results in a faster onvergene than for small problems and also, if properly adapted, for larger ones. 1 Introdution Big Data problems are putting a strong emphasis in using simple linear sparse models to handle large size and dimensional samples. This has made Lasso [1] and Elasti Net [] often the methods of hoie in large sale Mahine Learning. Both are usually stated in their unonstrained version of solving min Uλ,µ (β) = β 1 µ kxβ yk + kβk + λkβk1, (1) where for an N -size sample S = {(x1, y 1 ),..., (xn, y N )}, X is an N d data matrix ontaining as its rows the transposes of the d-dimensional features xn, y = (y 1,..., y N )t is the N 1 target vetor, β denotes the Lasso (when µ = 0) or Elasti Net model oeffiients, and the subsripts 1, denote the `1 and ` norms respetively. We will refer to problem (1) as λ-unonstrained Lasso (or λ-ul), writing then simply Uλ, or as (µ, λ)-unonstrained Elasti Net(or (µ, λ)-uen). It has an equivalent onstrained formulation min Cρ,µ (β) = β 1 µ kxβ yk + kβk s.t. kβk1 ρ. () Again, we will refer to problem () as ρ-cl, and write Cρ, or as (µ, ρ)-cen. Two algorithmi approahes stand out when solving (1), the algorithm [3] that uses yli oordinate desent and the FISTA algorithm [4], based on proximal optimization. Reently M. Jaggi [5] has shown the equivalene between With partial support from Spain s grants TIN P and S013/ICE-845 CASICAM-CM and also of the Ca tedra UAM ADIC in Data Siene and Mahine Learning. The authors also gratefully aknowledge the use of the failities of Centro de Computaio n Cientı fia (CCC) at UAM. The seond author is also supported by the FPU MEC grant AP
2 onstrained Lasso and some variants. This is relatively simple when going from Lasso to (the diretion we are interested in) but more involved the other way around. In [6] Jaggi s approah is refined to obtain a one way redution from onstrained Elasti Net to a squared hinge loss problem. As mentioned in [5], this opens the way for advanes in one problem to be translated in advanes into another and, while the appliation of solvers to Lasso is not addressed in [5], prior work in parallelizing s is leveraged in [6] to obtain a highly optimized and parallel solver for the Elasti Net and Lasso. In this paper we retake M. Jaggi s approah, first to simplify and refine the redutions in [5] and [6] and, then, to explore the effiieny of the solver LIB, when applied to Lasso. More preisely, our ontributions are A simple redution in Set. of Elasti Net to Lasso and a proof of the equivalene between λ-ul and ρ-cl. This is a known result but proofs are hard to find, so our simple argument may have a value in itself. A refinement in Set. 3 on M. Jaggi s arguments to redue ρ-cl to Nearest Point and ν-svc problems solvable using LIB. An experimental omparison in Set. 4 of NPP-LIB with and FISTA, showing NPP-LIB to be ompetitive with. The paper will lose with a brief disussion and pointers to further work. Unonstrained and Constrained Lasso and Elasti Net We show that λ-cl and ρ-ul and also (µ, λ)-uen and (µ, ρ)-cen have the same β solution for appropriate hoies of λ and ρ. We will first redue our disussion to the Lasso ase desribing how Elasti Net (EN) an be reast as a pure Lasso problem over an enlarged data matrix and target vetor. For this let s onsider in either problem (1) or () the (N + d) d matrix X and (N + d) 1 vetor Y defined as X t = (X t, µid ), Y t = (y t, 0td ) with Id the d d identity matrix and 0d the d dimensional 0 vetor. It is now easy to hek that kxβ yk + µkβk = kx β Yk and, thus, we an rewrite (1) or () as Lasso problems over an extended sample S = {(x1, y 1 ),..., (xn, y N ), ( µe1, 0),..., ( µed, 0)} of size N + d, where ei denotes the anonial vetors for Rd, for whih X and Y are the new data matrix and target vetor. Beause of this, in what follows we limit our disussion to UL and CL, pointing where needed the hanges to be made for UEN and CEN, and reverting to the (X, y) notations instead of (X, Y). Assuming λ fixed it is obvious that a minimizer βλ of λ-ul is also a minimizer for ρλ -CL with ρλ = kβλ k1. Next, if βlr is the solution of Linear Regression and ρlr = kβlr k1, the solution of ρ-cl is learly βlr for ρ ρlr, so we assume ρ < ρlr. Let s denote by βρ a minimum of the onvex problem ρ-cl; we shall 68
3 prove next that βρ solves λρ -UL with λρ = βρt X t (Xβρ y) /ρ. To do so, let e (β) = 1 kxβ yk and g(β) = kβk1 ρ, and define f (β) = max{e (β) eρ, g(β)}, where eρ = e (βρ ) is the optimal square error in ρ-cl. Then, sine f is onvex, the subgradient f (β 0 ) at any β 0 is f (β 0 ) = {γ e (β 0 ) + (1 γ)h : 0 γ 1, h g(β 0 ) = k k1 (βρ )}. Thus, as βρ minimizes f (any β 0 with kβ 0 k1 < ρ will have and error greater than eρ ), there is an hρ k k1 (βρ ) and γ, 0 γ 1, suh that 0 = γ e (βρ ) + (1 γ)hρ. But γ > 0, otherwise we would have hρ = 0, i.e., βρ = 0, ontraditing that kβρ k1 = ρ. Therefore, taking λρ = (1 γ)/γ, we have 0 = e (βρ ) + λρ hρ [e ( ) + λρ k k1 ] (βρ ) = Uλρ (βρ ), i.e., βρ minimizes λρ -UL. We finally derive a value for λρ by observing that βρt hρ = kβρ k1 = ρ and 0 = e (βρ ) + λρ hρ = X t rβρ + hρ λρ, where rβρ is just the residual (Xβρ y). As a result, λρ kβρ k1 = βρt hλρ = βρt X t rβρ λρ = βρt X t rβρ βρt X t rβρ =. kβρ k1 ρ For (µ, ρ)-cen, the orresponding λρ would be λρ = 3 (Y X βρ ) X βρ (y Xβρ ) Xβρ µkβρ k =. ρ ρ From Constrained Lasso to NPP The basi idea in [5] is to re-sale β = β/ρ and y = y/ρ to normalize ρ-cl into 1-CL and then to replae the `1 unit ball B1 with the simplex d = {α = (α1,..., αd ) Rd : 0 αi 1, d X αi = 1}, i=1 b with olumns by enlarging the initial data matrix X to an N d dimensional X d+ b b X = X and X = X, = 1,..., d. As a onsequene, the square error in b y k. Sine Xα b now lies in the the re-saled Lasso kx β y k beomes kxα b onvex hull C spanned by {X, 1 d}, finding an optimal αo is equivalent to finding the point in C losest to y, i.e., solving the nearest point problem (NPP) between the N -dimensional onvex hull C and the singleton {y }. In turn, NPP an be saled [7] into an equivalent linear ν-svc problem [8] with ν = /(d+1), over the sample S = {S+, S }, where S+ = {X 1,..., X d, X 1,..., X d } and S = {y }. This problem an be solved using the LIB library [9]. The optimal ρ-cl solution βρ is reovered as follows: first we obtain the optimal NPP oeffiients αo by saling the ν- solution γ o as αo = (d + 1)γ o, then o we ompute (β ρ )i = αio αi+d and finally we re-sale again βρ = ρβ ρ. Finally, hanges for (µ, ρ)-cen are straightforward, sine we only have to add the d extra dimensions µe and µe to the olumn vetors X. 69
4 4 Numerial Experiments In this setion we will ompare the performane of the LIB approah for solving ρ-cl with two well known, state-of-the-art algorithms for λ-ul, FISTA and. We first disuss their expeted omplexity. FISTA (Fast ISTA; [4]) ombines the basi iterations in Iterative Shrinkage Thresholding Algorithm (ISTA) with a Nesterov aeleration step. Assuming that the ovariane X t X is preomputed at a fixed initial ost O(N d ), the ost per iteration of FISTA is O(d ), i.e., that of omputing X t Xβ. If only m omponents of β are non zero, the iteration ost is O(dm). On the other hand, performs yli oordinate subgradient desent on the λ-ul ost funtion. arefully manages the ovariane omputations X j X k ensuring that there is a ost O(N d) only the first time they are performed at a given oordinate k and that afterwards their ost is just O(d). Note that an iteration in just hanges one oordinate while in FISTA it hanges d omponents. Finally, the ν-svc solver in LIB performs SMO-like iterations that hange two oordinates, the pair that most violates the ν-svc KKT onditions. The ost per iteration is also O(d) plus the time needed to ompute the required dot produts, i.e., linear kernel operations. LIB builds the kernel matrix making no assumptions on the partiular struture of the data. This may lead to eventually ompute 4d dot produts (without onsidering the last olumn) even though only d are atually needed, sine the d d dimensional linear kernel matrix X X t is made of four d d bloks with the ovariane matrix C = XX t in the two diagonal bloks and C in the other two. This an be ertainly ontrolled but in our experiments we have diretly used LIB, without any attempt to optimize running times exploiting the struture of the kernel matrix. We ompare next the number of iterations and running times that FISTA, and LIB need to solve equivalent λ-ul and ρ-cl problems. We will do so over four datasets from the UCI olletion, namely the relatively small prostate and housing, and the larger year and tsan. As it is well known, training omplexity greatly depends on λ. We will onsider three possible λ values for UL: an optimal λ obtained aording to the regularization path proedure of [3], a smaller λ / value (that should result in longer training) and a striter penalty λ value. The optimal λ values for the problems onsidered are , , and respetively. The orresponding ρ parameters are omputed as ρ = kβλ k1, with βλ the optimal solution for λ-ul. To make a balaned omparison, for eah λ and dataset we first make a long run of eah method M so that it onverges to a βm that we take as the k optimum. We then ompare for eah M the evolution of fm (βm ) fm (βm ), k with βm the oeffiients at the k-th iteration of method M, until it is smaller than a threshold that we take as 10 1 for prostate and housing and 10 6 for year and tsan. Table 1 shows in olumns to 4 the number of iterations that eah method requires to arrive at the threshold. As it an be seen, LIB is the fastest on this aount, with in seond plae and FISTA a more distant third (for a proper omparison a FISTA iterate is made to orrespond to 70
5 Iterations Dataset Time (ms) LIB FISTA LIB (λ ) prostate prostate ( λ ) prostate (λ /) ,3 1, housing (λ ) housing ( λ ) housing (λ /) , ,538 5,538 3, year (λ ) year ( λ ) year (λ /) ,584 6,13 6,393 8,550 8,460 8,640 1,58.9 1, , tsan (λ ) tsan ( λ ) tsan (λ /) ,390 78,456 53, , ,744 13,480 3, , , ,83.9 6, ,173.7 Table 1: Iterations and running times. d iterates of LIB and ). Columns 5 and 6 give the times required by LIB and ; we omit FISTA s times as they are not ompetitive (at least under the implementation we used). We use the LIB and implementations in the Sikit Python library, whih both have a ompiled C ore, so we may expet time omparisons to be broadly homogeneous. Even without onsidering LIB s overhead when omputing a d d kernel matrix, it is learly faster on prostate and housing but not so on the other two datasets. However, it turns out that LIB indeed omputes about 4 times more dot produts than needed, whih suggests that the running time of a LIB version properly adapted for ρ-cl should have running times about a fourth of those reported here, outperforming then. This behavior is further illustrated in Fig. 1 that depits, for housing and tsan with λ, the number of iterations and running times required to reah the threshold. 5 Disussion and Conlusions an be onsidered the urrent state of the art to solve the Lasso problem. M. Jaggi s reent observation that ρ-cl an be redued to ν-svc opens the way to the appliation of methods. In this work we have shown using four examples how the ν-svc option of LIB is faster than for small problems and pointed out how adapted versions ould also beat it on larger problems. Devising suh an adaptation is thus a first further line of work. Moreover, Lasso is at the ore of many other problems in onvex regularization, suh as Fused Lasso, wavelet smoothing or trend filtering, urrently solved using speialized algorithms. A ν-svc approah ould provide for them the same faster onvergene that we have illustrated here for standard Lasso. Furthermore, the ν-svc training ould be speed up, as suggested in [5], using the sreening rules available for Lasso [10] in order to remove olumns of the data matrix. We are working on these and other related questions. 71
6 Time - housing Iterations - housing Objetive 100 FISTA ,000 6, ,000 Iterations - tsan Time - tsan FISTA 100 Objetive 0. Time (ms) Iteration , Iteration ,000 10,000 15,000 Time (ms) Fig. 1: Results for housing and tsan. Referenes [1] R. Tibshirani. Regression Shrinkage and Seletion Via the Lasso. Journal of the Royal Statistial Soiety and Series B, 58:67 88, [] H. Zou and T. Hastie. Regularization and variable seletion via the elasti net. Journal of the Royal Statistial Soiety: Series B (Statistial Methodology), 67():301 30, 005. [3] J. H. Friedman, T. Hastie, and R. Tibshirani. Regularization Paths for Generalized Linear Models via Coordinate Desent. Journal of Statistial Software, 33(1):1, 010. [4] A. Bek and M. Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Img. Si., (1):183 0, 009. [5] M. Jaggi. An Equivalene between the Lasso and Support Vetor Mahines. In Regularization, Optimization, Kernels, and Support Vetor Mahines. CRC Press, 014. [6] Q. Zhou, W. Chen, S. Song, J. R. Gardner, K. Q. Weinberger, and Y. Chen. A Redution of the Elasti Net to Support Vetor Mahines with an Appliation to GPU Computing. Tehnial Report arxiv: [stat.ml], 014. [7] J. Lo pez, A. Barbero, and J.R. Dorronsoro. Clipping Algorithms for Solving the Nearest Point Problem over Redued Convex Hulls. Pattern Reognition, 44(3): , 011. [8] C.-C. Chang and C.-J. Lin. Training ν-support Vetor Classifiers: Theory and Algorithms. Neural Computation, 13(9): , 001. [9] C.-C. Chang and C.-J. Lin. LIB: a Library for Support Vetor Mahines. http: // [10] J. Wang, J. Zhou, P. Wonka, and J. Ye. Lasso sreening rules via dual polytope projetion. In C.J.C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger, editors, Advanes in Neural Information Proessing Systems 6, pages Curran Assoiates, In.,
Complexity of Regularization RBF Networks
Complexity of Regularization RBF Networks Mark A Kon Department of Mathematis and Statistis Boston University Boston, MA 02215 mkon@buedu Leszek Plaskota Institute of Applied Mathematis University of Warsaw
More informationModel-based mixture discriminant analysis an experimental study
Model-based mixture disriminant analysis an experimental study Zohar Halbe and Mayer Aladjem Department of Eletrial and Computer Engineering, Ben-Gurion University of the Negev P.O.Box 653, Beer-Sheva,
More informationA NETWORK SIMPLEX ALGORITHM FOR THE MINIMUM COST-BENEFIT NETWORK FLOW PROBLEM
NETWORK SIMPLEX LGORITHM FOR THE MINIMUM COST-BENEFIT NETWORK FLOW PROBLEM Cen Çalışan, Utah Valley University, 800 W. University Parway, Orem, UT 84058, 801-863-6487, en.alisan@uvu.edu BSTRCT The minimum
More informationBilinear Formulated Multiple Kernel Learning for Multi-class Classification Problem
Bilinear Formulated Multiple Kernel Learning for Multi-lass Classifiation Problem Takumi Kobayashi and Nobuyuki Otsu National Institute of Advaned Industrial Siene and Tehnology, -- Umezono, Tsukuba, Japan
More informationmax min z i i=1 x j k s.t. j=1 x j j:i T j
AM 221: Advaned Optimization Spring 2016 Prof. Yaron Singer Leture 22 April 18th 1 Overview In this leture, we will study the pipage rounding tehnique whih is a deterministi rounding proedure that an be
More informationTaste for variety and optimum product diversity in an open economy
Taste for variety and optimum produt diversity in an open eonomy Javier Coto-Martínez City University Paul Levine University of Surrey Otober 0, 005 María D.C. Garía-Alonso University of Kent Abstrat We
More informationThe Effectiveness of the Linear Hull Effect
The Effetiveness of the Linear Hull Effet S. Murphy Tehnial Report RHUL MA 009 9 6 Otober 009 Department of Mathematis Royal Holloway, University of London Egham, Surrey TW0 0EX, England http://www.rhul.a.uk/mathematis/tehreports
More informationMaximum Entropy and Exponential Families
Maximum Entropy and Exponential Families April 9, 209 Abstrat The goal of this note is to derive the exponential form of probability distribution from more basi onsiderations, in partiular Entropy. It
More informationThe Laws of Acceleration
The Laws of Aeleration The Relationships between Time, Veloity, and Rate of Aeleration Copyright 2001 Joseph A. Rybzyk Abstrat Presented is a theory in fundamental theoretial physis that establishes the
More informationA new initial search direction for nonlinear conjugate gradient method
International Journal of Mathematis Researh. ISSN 0976-5840 Volume 6, Number 2 (2014), pp. 183 190 International Researh Publiation House http://www.irphouse.om A new initial searh diretion for nonlinear
More informationA Unified View on Multi-class Support Vector Classification Supplement
Journal of Mahine Learning Researh??) Submitted 7/15; Published?/?? A Unified View on Multi-lass Support Vetor Classifiation Supplement Ürün Doğan Mirosoft Researh Tobias Glasmahers Institut für Neuroinformatik
More informationSupplementary Materials
Supplementary Materials Neural population partitioning and a onurrent brain-mahine interfae for sequential motor funtion Maryam M. Shanehi, Rollin C. Hu, Marissa Powers, Gregory W. Wornell, Emery N. Brown
More informationMillennium Relativity Acceleration Composition. The Relativistic Relationship between Acceleration and Uniform Motion
Millennium Relativity Aeleration Composition he Relativisti Relationship between Aeleration and niform Motion Copyright 003 Joseph A. Rybzyk Abstrat he relativisti priniples developed throughout the six
More informationCounting Idempotent Relations
Counting Idempotent Relations Beriht-Nr. 2008-15 Florian Kammüller ISSN 1436-9915 2 Abstrat This artile introdues and motivates idempotent relations. It summarizes haraterizations of idempotents and their
More informationAdvanced Computational Fluid Dynamics AA215A Lecture 4
Advaned Computational Fluid Dynamis AA5A Leture 4 Antony Jameson Winter Quarter,, Stanford, CA Abstrat Leture 4 overs analysis of the equations of gas dynamis Contents Analysis of the equations of gas
More informationControl Theory association of mathematics and engineering
Control Theory assoiation of mathematis and engineering Wojieh Mitkowski Krzysztof Oprzedkiewiz Department of Automatis AGH Univ. of Siene & Tehnology, Craow, Poland, Abstrat In this paper a methodology
More informationAn Adaptive Optimization Approach to Active Cancellation of Repeated Transient Vibration Disturbances
An aptive Optimization Approah to Ative Canellation of Repeated Transient Vibration Disturbanes David L. Bowen RH Lyon Corp / Aenteh, 33 Moulton St., Cambridge, MA 138, U.S.A., owen@lyonorp.om J. Gregory
More informationHankel Optimal Model Order Reduction 1
Massahusetts Institute of Tehnology Department of Eletrial Engineering and Computer Siene 6.245: MULTIVARIABLE CONTROL SYSTEMS by A. Megretski Hankel Optimal Model Order Redution 1 This leture overs both
More information10.5 Unsupervised Bayesian Learning
The Bayes Classifier Maximum-likelihood methods: Li Yu Hongda Mao Joan Wang parameter vetor is a fixed but unknown value Bayes methods: parameter vetor is a random variable with known prior distribution
More informationA model for measurement of the states in a coupled-dot qubit
A model for measurement of the states in a oupled-dot qubit H B Sun and H M Wiseman Centre for Quantum Computer Tehnology Centre for Quantum Dynamis Griffith University Brisbane 4 QLD Australia E-mail:
More informationThe Hanging Chain. John McCuan. January 19, 2006
The Hanging Chain John MCuan January 19, 2006 1 Introdution We onsider a hain of length L attahed to two points (a, u a and (b, u b in the plane. It is assumed that the hain hangs in the plane under a
More informationSURFACE WAVES OF NON-RAYLEIGH TYPE
SURFACE WAVES OF NON-RAYLEIGH TYPE by SERGEY V. KUZNETSOV Institute for Problems in Mehanis Prosp. Vernadskogo, 0, Mosow, 75 Russia e-mail: sv@kuznetsov.msk.ru Abstrat. Existene of surfae waves of non-rayleigh
More informationEstimating the probability law of the codelength as a function of the approximation error in image compression
Estimating the probability law of the odelength as a funtion of the approximation error in image ompression François Malgouyres Marh 7, 2007 Abstrat After some reolletions on ompression of images using
More informationINTERNATIONAL JOURNAL OF CIVIL AND STRUCTURAL ENGINEERING Volume 2, No 4, 2012
INTERNATIONAL JOURNAL OF CIVIL AND STRUCTURAL ENGINEERING Volume, No 4, 01 Copyright 010 All rights reserved Integrated Publishing servies Researh artile ISSN 0976 4399 Strutural Modelling of Stability
More informationLecture 7: Sampling/Projections for Least-squares Approximation, Cont. 7 Sampling/Projections for Least-squares Approximation, Cont.
Stat60/CS94: Randomized Algorithms for Matries and Data Leture 7-09/5/013 Leture 7: Sampling/Projetions for Least-squares Approximation, Cont. Leturer: Mihael Mahoney Sribe: Mihael Mahoney Warning: these
More informationA Queueing Model for Call Blending in Call Centers
A Queueing Model for Call Blending in Call Centers Sandjai Bhulai and Ger Koole Vrije Universiteit Amsterdam Faulty of Sienes De Boelelaan 1081a 1081 HV Amsterdam The Netherlands E-mail: {sbhulai, koole}@s.vu.nl
More informationCoding for Random Projections and Approximate Near Neighbor Search
Coding for Random Projetions and Approximate Near Neighbor Searh Ping Li Department of Statistis & Biostatistis Department of Computer Siene Rutgers University Pisataay, NJ 8854, USA pingli@stat.rutgers.edu
More informationA variant of Coppersmith s Algorithm with Improved Complexity and Efficient Exhaustive Search
A variant of Coppersmith s Algorithm with Improved Complexity and Effiient Exhaustive Searh Jean-Sébastien Coron 1, Jean-Charles Faugère 2, Guénaël Renault 2, and Rina Zeitoun 2,3 1 University of Luxembourg
More informationDanielle Maddix AA238 Final Project December 9, 2016
Struture and Parameter Learning in Bayesian Networks with Appliations to Prediting Breast Caner Tumor Malignany in a Lower Dimension Feature Spae Danielle Maddix AA238 Final Projet Deember 9, 2016 Abstrat
More informationModeling of Threading Dislocation Density Reduction in Heteroepitaxial Layers
A. E. Romanov et al.: Threading Disloation Density Redution in Layers (II) 33 phys. stat. sol. (b) 99, 33 (997) Subjet lassifiation: 6.72.C; 68.55.Ln; S5.; S5.2; S7.; S7.2 Modeling of Threading Disloation
More informationRobust Flight Control Design for a Turn Coordination System with Parameter Uncertainties
Amerian Journal of Applied Sienes 4 (7): 496-501, 007 ISSN 1546-939 007 Siene Publiations Robust Flight ontrol Design for a urn oordination System with Parameter Unertainties 1 Ari Legowo and Hiroshi Okubo
More informationREFINED UPPER BOUNDS FOR THE LINEAR DIOPHANTINE PROBLEM OF FROBENIUS. 1. Introduction
Version of 5/2/2003 To appear in Advanes in Applied Mathematis REFINED UPPER BOUNDS FOR THE LINEAR DIOPHANTINE PROBLEM OF FROBENIUS MATTHIAS BECK AND SHELEMYAHU ZACKS Abstrat We study the Frobenius problem:
More informationAn I-Vector Backend for Speaker Verification
An I-Vetor Bakend for Speaker Verifiation Patrik Kenny, 1 Themos Stafylakis, 1 Jahangir Alam, 1 and Marel Kokmann 2 1 CRIM, Canada, {patrik.kenny, themos.stafylakis, jahangir.alam}@rim.a 2 VoieTrust, Canada,
More informationOptimization of Statistical Decisions for Age Replacement Problems via a New Pivotal Quantity Averaging Approach
Amerian Journal of heoretial and Applied tatistis 6; 5(-): -8 Published online January 7, 6 (http://www.sienepublishinggroup.om/j/ajtas) doi:.648/j.ajtas.s.65.4 IN: 36-8999 (Print); IN: 36-96 (Online)
More informationScalable Positivity Preserving Model Reduction Using Linear Energy Functions
Salable Positivity Preserving Model Redution Using Linear Energy Funtions Sootla, Aivar; Rantzer, Anders Published in: IEEE 51st Annual Conferene on Deision and Control (CDC), 2012 DOI: 10.1109/CDC.2012.6427032
More informationA Spatiotemporal Approach to Passive Sound Source Localization
A Spatiotemporal Approah Passive Sound Soure Loalization Pasi Pertilä, Mikko Parviainen, Teemu Korhonen and Ari Visa Institute of Signal Proessing Tampere University of Tehnology, P.O.Box 553, FIN-330,
More informationModeling of discrete/continuous optimization problems: characterization and formulation of disjunctions and their relaxations
Computers and Chemial Engineering (00) 4/448 www.elsevier.om/loate/omphemeng Modeling of disrete/ontinuous optimization problems: haraterization and formulation of disjuntions and their relaxations Aldo
More informationNormative and descriptive approaches to multiattribute decision making
De. 009, Volume 8, No. (Serial No.78) China-USA Business Review, ISSN 57-54, USA Normative and desriptive approahes to multiattribute deision making Milan Terek (Department of Statistis, University of
More informationLikelihood-confidence intervals for quantiles in Extreme Value Distributions
Likelihood-onfidene intervals for quantiles in Extreme Value Distributions A. Bolívar, E. Díaz-Franés, J. Ortega, and E. Vilhis. Centro de Investigaión en Matemátias; A.P. 42, Guanajuato, Gto. 36; Méxio
More informationWhat s New in ChemSep TM 6.8
What s New in ChemSep TM 6.8 January 2011 (Updated Marh 2011) Harry Kooijman and Ross Taylor In this doument we identify and desribe the most important new features in ChemSep. 1. New: GUI an diretly load
More informationStabilization of the Precision Positioning Stage Working in the Vacuum Environment by Using the Disturbance Observer
Proeedings of the 4th IIAE International Conferene on Industrial Appliation Engineering 216 Stabilization of the Preision Positioning Stage Working in the Vauum Environment by Using the Disturbane Observer
More informationA Characterization of Wavelet Convergence in Sobolev Spaces
A Charaterization of Wavelet Convergene in Sobolev Spaes Mark A. Kon 1 oston University Louise Arakelian Raphael Howard University Dediated to Prof. Robert Carroll on the oasion of his 70th birthday. Abstrat
More informationConvergence of reinforcement learning with general function approximators
Convergene of reinforement learning with general funtion approximators assilis A. Papavassiliou and Stuart Russell Computer Siene Division, U. of California, Berkeley, CA 94720-1776 fvassilis,russellg@s.berkeley.edu
More informationOn the Complexity of the Weighted Fused Lasso
ON THE COMPLEXITY OF THE WEIGHTED FUSED LASSO On the Compleity of the Weighted Fused Lasso José Bento jose.bento@b.edu Ralph Furmaniak rf@am.org Surjyendu Ray rays@b.edu Abstrat The solution path of the
More informationEXACT TRAVELLING WAVE SOLUTIONS FOR THE GENERALIZED KURAMOTO-SIVASHINSKY EQUATION
Journal of Mathematial Sienes: Advanes and Appliations Volume 3, 05, Pages -3 EXACT TRAVELLING WAVE SOLUTIONS FOR THE GENERALIZED KURAMOTO-SIVASHINSKY EQUATION JIAN YANG, XIAOJUAN LU and SHENGQIANG TANG
More informationMATHEMATICAL AND NUMERICAL BASIS OF BINARY ALLOY SOLIDIFICATION MODELS WITH SUBSTITUTE THERMAL CAPACITY. PART II
Journal of Applied Mathematis and Computational Mehanis 2014, 13(2), 141-147 MATHEMATICA AND NUMERICA BAI OF BINARY AOY OIDIFICATION MODE WITH UBTITUTE THERMA CAPACITY. PART II Ewa Węgrzyn-krzypzak 1,
More informationWord of Mass: The Relationship between Mass Media and Word-of-Mouth
Word of Mass: The Relationship between Mass Media and Word-of-Mouth Roman Chuhay Preliminary version Marh 6, 015 Abstrat This paper studies the optimal priing and advertising strategies of a firm in the
More informationCSC2515 Winter 2015 Introduc3on to Machine Learning. Lecture 5: Clustering, mixture models, and EM
CSC2515 Winter 2015 Introdu3on to Mahine Learning Leture 5: Clustering, mixture models, and EM All leture slides will be available as.pdf on the ourse website: http://www.s.toronto.edu/~urtasun/ourses/csc2515/
More informationPerturbation Analyses for the Cholesky Factorization with Backward Rounding Errors
Perturbation Analyses for the holesky Fatorization with Bakward Rounding Errors Xiao-Wen hang Shool of omputer Siene, MGill University, Montreal, Quebe, anada, H3A A7 Abstrat. This paper gives perturbation
More informationComparison of Alternative Equivalent Circuits of Induction Motor with Real Machine Data
Comparison of Alternative Equivalent Ciruits of Indution Motor with Real Mahine Data J. radna, J. auer, S. Fligl and V. Hlinovsky Abstrat The algorithms based on separated ontrol of the motor flux and
More informationThe Second Postulate of Euclid and the Hyperbolic Geometry
1 The Seond Postulate of Eulid and the Hyperboli Geometry Yuriy N. Zayko Department of Applied Informatis, Faulty of Publi Administration, Russian Presidential Aademy of National Eonomy and Publi Administration,
More informationLECTURE NOTES FOR , FALL 2004
LECTURE NOTES FOR 18.155, FALL 2004 83 12. Cone support and wavefront set In disussing the singular support of a tempered distibution above, notie that singsupp(u) = only implies that u C (R n ), not as
More informationLecture 3 - Lorentz Transformations
Leture - Lorentz Transformations A Puzzle... Example A ruler is positioned perpendiular to a wall. A stik of length L flies by at speed v. It travels in front of the ruler, so that it obsures part of the
More informationAverage Rate Speed Scaling
Average Rate Speed Saling Nikhil Bansal David P. Bunde Ho-Leung Chan Kirk Pruhs May 2, 2008 Abstrat Speed saling is a power management tehnique that involves dynamially hanging the speed of a proessor.
More informationAn iterative least-square method suitable for solving large sparse matrices
An iteratie least-square method suitable for soling large sparse matries By I. M. Khabaza The purpose of this paper is to report on the results of numerial experiments with an iteratie least-square method
More informationDynamic Screening: Accelerating First-Order Algorithms for the Lasso and Group-Lasso
Dynami Sreening: Aelerating First-Order Algorithms for the Lasso and Group-Lasso Antoine Bonnefoy, Valentin Emiya, Liva Ralaivola, Rémi Gribonval To ite this version: Antoine Bonnefoy, Valentin Emiya,
More informationWave Propagation through Random Media
Chapter 3. Wave Propagation through Random Media 3. Charateristis of Wave Behavior Sound propagation through random media is the entral part of this investigation. This hapter presents a frame of referene
More informationProbabilistic Graphical Models
Probabilisti Graphial Models David Sontag New York University Leture 12, April 19, 2012 Aknowledgement: Partially based on slides by Eri Xing at CMU and Andrew MCallum at UMass Amherst David Sontag (NYU)
More information7 Max-Flow Problems. Business Computing and Operations Research 608
7 Max-Flow Problems Business Computing and Operations Researh 68 7. Max-Flow Problems In what follows, we onsider a somewhat modified problem onstellation Instead of osts of transmission, vetor now indiates
More informationNonreversibility of Multiple Unicast Networks
Nonreversibility of Multiple Uniast Networks Randall Dougherty and Kenneth Zeger September 27, 2005 Abstrat We prove that for any finite direted ayli network, there exists a orresponding multiple uniast
More informationBäcklund Transformations: Some Old and New Perspectives
Bäklund Transformations: Some Old and New Perspetives C. J. Papahristou *, A. N. Magoulas ** * Department of Physial Sienes, Helleni Naval Aademy, Piraeus 18539, Greee E-mail: papahristou@snd.edu.gr **
More informationMeasuring & Inducing Neural Activity Using Extracellular Fields I: Inverse systems approach
Measuring & Induing Neural Ativity Using Extraellular Fields I: Inverse systems approah Keith Dillon Department of Eletrial and Computer Engineering University of California San Diego 9500 Gilman Dr. La
More informationWeighted K-Nearest Neighbor Revisited
Weighted -Nearest Neighbor Revisited M. Biego University of Verona Verona, Italy Email: manuele.biego@univr.it M. Loog Delft University of Tehnology Delft, The Netherlands Email: m.loog@tudelft.nl Abstrat
More informationA NONLILEAR CONTROLLER FOR SHIP AUTOPILOTS
Vietnam Journal of Mehanis, VAST, Vol. 4, No. (), pp. A NONLILEAR CONTROLLER FOR SHIP AUTOPILOTS Le Thanh Tung Hanoi University of Siene and Tehnology, Vietnam Abstrat. Conventional ship autopilots are
More informationMoments and Wavelets in Signal Estimation
Moments and Wavelets in Signal Estimation Edward J. Wegman 1 Center for Computational Statistis George Mason University Hung T. Le 2 International usiness Mahines Abstrat: The problem of generalized nonparametri
More informationAssessing the Performance of a BCI: A Task-Oriented Approach
Assessing the Performane of a BCI: A Task-Oriented Approah B. Dal Seno, L. Mainardi 2, M. Matteui Department of Eletronis and Information, IIT-Unit, Politenio di Milano, Italy 2 Department of Bioengineering,
More informationOn the Licensing of Innovations under Strategic Delegation
On the Liensing of Innovations under Strategi Delegation Judy Hsu Institute of Finanial Management Nanhua University Taiwan and X. Henry Wang Department of Eonomis University of Missouri USA Abstrat This
More informationOn the application of the spectral projected gradient method in image segmentation
Noname manusript No. will be inserted by the editor) On the appliation of the spetral projeted gradient method in image segmentation Laura Antonelli Valentina De Simone Daniela di Serafino May 22, 2015
More informationThe gravitational phenomena without the curved spacetime
The gravitational phenomena without the urved spaetime Mirosław J. Kubiak Abstrat: In this paper was presented a desription of the gravitational phenomena in the new medium, different than the urved spaetime,
More informationAdaptive neuro-fuzzy inference system-based controllers for smart material actuator modelling
Adaptive neuro-fuzzy inferene system-based ontrollers for smart material atuator modelling T L Grigorie and R M Botez Éole de Tehnologie Supérieure, Montréal, Quebe, Canada The manusript was reeived on
More informationMOLECULAR ORBITAL THEORY- PART I
5.6 Physial Chemistry Leture #24-25 MOLECULAR ORBITAL THEORY- PART I At this point, we have nearly ompleted our rash-ourse introdution to quantum mehanis and we re finally ready to deal with moleules.
More informationSufficient Conditions for a Flexible Manufacturing System to be Deadlocked
Paper 0, INT 0 Suffiient Conditions for a Flexile Manufaturing System to e Deadloked Paul E Deering, PhD Department of Engineering Tehnology and Management Ohio University deering@ohioedu Astrat In reent
More information23.1 Tuning controllers, in the large view Quoting from Section 16.7:
Lesson 23. Tuning a real ontroller - modeling, proess identifiation, fine tuning 23.0 Context We have learned to view proesses as dynami systems, taking are to identify their input, intermediate, and output
More informationThe Influences of Smooth Approximation Functions for SPTSVM
The Influenes of Smooth Approximation Funtions for SPTSVM Xinxin Zhang Liaoheng University Shool of Mathematis Sienes Liaoheng, 5059 P.R. China ldzhangxin008@6.om Liya Fan Liaoheng University Shool of
More informationVariation Based Online Travel Time Prediction Using Clustered Neural Networks
Variation Based Online Travel Time Predition Using lustered Neural Networks Jie Yu, Gang-Len hang, H.W. Ho and Yue Liu Abstrat-This paper proposes a variation-based online travel time predition approah
More informationIDENTIFICATION AND CONTROL OF ACOUSTIC RADIATION MODES
IDENTIFICATION AND CONTROL OF ACOUSTIC RADIATION MODES Arthur P. Berkhoff University of Twente, Faulty of Eletrial Engineering, P.O. Box 217, 7 AE Enshede, The Netherlands email: a.p.berkhoff@el.utwente.nl
More informationDiscrete Bessel functions and partial difference equations
Disrete Bessel funtions and partial differene equations Antonín Slavík Charles University, Faulty of Mathematis and Physis, Sokolovská 83, 186 75 Praha 8, Czeh Republi E-mail: slavik@karlin.mff.uni.z Abstrat
More informationRobust Recovery of Signals From a Structured Union of Subspaces
Robust Reovery of Signals From a Strutured Union of Subspaes 1 Yonina C. Eldar, Senior Member, IEEE and Moshe Mishali, Student Member, IEEE arxiv:87.4581v2 [nlin.cg] 3 Mar 29 Abstrat Traditional sampling
More informationOrdered fields and the ultrafilter theorem
F U N D A M E N T A MATHEMATICAE 59 (999) Ordered fields and the ultrafilter theorem by R. B e r r (Dortmund), F. D e l o n (Paris) and J. S h m i d (Dortmund) Abstrat. We prove that on the basis of ZF
More informationMaximum Likelihood Multipath Estimation in Comparison with Conventional Delay Lock Loops
Maximum Likelihood Multipath Estimation in Comparison with Conventional Delay Lok Loops Mihael Lentmaier and Bernhard Krah, German Aerospae Center (DLR) BIOGRAPY Mihael Lentmaier reeived the Dipl.-Ing.
More informationA simple expression for radial distribution functions of pure fluids and mixtures
A simple expression for radial distribution funtions of pure fluids and mixtures Enrio Matteoli a) Istituto di Chimia Quantistia ed Energetia Moleolare, CNR, Via Risorgimento, 35, 56126 Pisa, Italy G.
More informationParallel disrete-event simulation is an attempt to speed-up the simulation proess through the use of multiple proessors. In some sense parallel disret
Exploiting intra-objet dependenies in parallel simulation Franeso Quaglia a;1 Roberto Baldoni a;2 a Dipartimento di Informatia e Sistemistia Universita \La Sapienza" Via Salaria 113, 198 Roma, Italy Abstrat
More informationarxiv: v2 [math.pr] 9 Dec 2016
Omnithermal Perfet Simulation for Multi-server Queues Stephen B. Connor 3th Deember 206 arxiv:60.0602v2 [math.pr] 9 De 206 Abstrat A number of perfet simulation algorithms for multi-server First Come First
More informationThe Electromagnetic Radiation and Gravity
International Journal of Theoretial and Mathematial Physis 016, 6(3): 93-98 DOI: 10.593/j.ijtmp.0160603.01 The Eletromagneti Radiation and Gravity Bratianu Daniel Str. Teiului Nr. 16, Ploiesti, Romania
More informationA new method of measuring similarity between two neutrosophic soft sets and its application in pattern recognition problems
Neutrosophi Sets and Systems, Vol. 8, 05 63 A new method of measuring similarity between two neutrosophi soft sets and its appliation in pattern reognition problems Anjan Mukherjee, Sadhan Sarkar, Department
More informationEffects of Vane Sweep on Fan-Wake/Outlet-Guide-Vane Interaction Broadband Noise
Effets of Vane Sweep on Fan-Wake/Outlet-Guide-Vane Interation Broadband Noise Hongbin Ju* GE Global Researh Center, One Researh Cirle, Niskayuna, NY. 09 A method is developed for prediting broadband noise
More informationSensitivity Analysis in Markov Networks
Sensitivity Analysis in Markov Networks Hei Chan and Adnan Darwihe Computer Siene Department University of California, Los Angeles Los Angeles, CA 90095 {hei,darwihe}@s.ula.edu Abstrat This paper explores
More informationLightpath routing for maximum reliability in optical mesh networks
Vol. 7, No. 5 / May 2008 / JOURNAL OF OPTICAL NETWORKING 449 Lightpath routing for maximum reliability in optial mesh networks Shengli Yuan, 1, * Saket Varma, 2 and Jason P. Jue 2 1 Department of Computer
More informationExploring the feasibility of on-site earthquake early warning using close-in records of the 2007 Noto Hanto earthquake
Exploring the feasibility of on-site earthquake early warning using lose-in reords of the 2007 Noto Hanto earthquake Yih-Min Wu 1 and Hiroo Kanamori 2 1. Department of Geosienes, National Taiwan University,
More informationJAST 2015 M.U.C. Women s College, Burdwan ISSN a peer reviewed multidisciplinary research journal Vol.-01, Issue- 01
JAST 05 M.U.C. Women s College, Burdwan ISSN 395-353 -a peer reviewed multidisiplinary researh journal Vol.-0, Issue- 0 On Type II Fuzzy Parameterized Soft Sets Pinaki Majumdar Department of Mathematis,
More informationLOGISTIC REGRESSION IN DEPRESSION CLASSIFICATION
LOGISIC REGRESSIO I DEPRESSIO CLASSIFICAIO J. Kual,. V. ran, M. Bareš KSE, FJFI, CVU v Praze PCP, CS, 3LF UK v Praze Abstrat Well nown logisti regression and the other binary response models an be used
More informationONLINE APPENDICES for Cost-Effective Quality Assurance in Crowd Labeling
ONLINE APPENDICES for Cost-Effetive Quality Assurane in Crowd Labeling Jing Wang Shool of Business and Management Hong Kong University of Siene and Tehnology Clear Water Bay Kowloon Hong Kong jwang@usthk
More informationV. Interacting Particles
V. Interating Partiles V.A The Cumulant Expansion The examples studied in the previous setion involve non-interating partiles. It is preisely the lak of interations that renders these problems exatly solvable.
More informationEE 321 Project Spring 2018
EE 21 Projet Spring 2018 This ourse projet is intended to be an individual effort projet. The student is required to omplete the work individually, without help from anyone else. (The student may, however,
More informationTHE METHOD OF SECTIONING WITH APPLICATION TO SIMULATION, by Danie 1 Brent ~~uffman'i
THE METHOD OF SECTIONING '\ WITH APPLICATION TO SIMULATION, I by Danie 1 Brent ~~uffman'i Thesis submitted to the Graduate Faulty of the Virginia Polytehni Institute and State University in partial fulfillment
More informationRelativity fundamentals explained well (I hope) Walter F. Smith, Haverford College
Relativity fundamentals explained well (I hope) Walter F. Smith, Haverford College 3-14-06 1 Propagation of waves through a medium As you ll reall from last semester, when the speed of sound is measured
More informationOn the Quantum Theory of Radiation.
Physikalishe Zeitshrift, Band 18, Seite 121-128 1917) On the Quantum Theory of Radiation. Albert Einstein The formal similarity between the hromati distribution urve for thermal radiation and the Maxwell
More informationSYNTHETIC APERTURE IMAGING OF DIRECTION AND FREQUENCY DEPENDENT REFLECTIVITIES
SYNTHETIC APERTURE IMAGING OF DIRECTION AND FREQUENCY DEPENDENT REFLECTIVITIES LILIANA BORCEA, MIGUEL MOSCOSO, GEORGE PAPANICOLAOU, AND CHRYSOULA TSOGKA Abstrat. We introdue a syntheti aperture imaging
More informationMODELING MATTER AT NANOSCALES. 4. Introduction to quantum treatments Eigenvectors and eigenvalues of a matrix
MODELING MATTER AT NANOSCALES 4 Introdution to quantum treatments 403 Eigenvetors and eigenvalues of a matrix Simultaneous equations in the variational method The problem of simultaneous equations in the
More informationError Bounds for Context Reduction and Feature Omission
Error Bounds for Context Redution and Feature Omission Eugen Bek, Ralf Shlüter, Hermann Ney,2 Human Language Tehnology and Pattern Reognition, Computer Siene Department RWTH Aahen University, Ahornstr.
More information