DEPARTMENT OF ECONOMETRICS AND BUSINESS STATISTICS
|
|
- Phillip Hood
- 5 years ago
- Views:
Transcription
1 ISSN X AUSTRALIA DEPARTMENT OF ECONOMETRICS AND BUSINESS STATISTICS An Iproved Method for Bandwidth Selection When Estiating ROC Curves Peter G Hall and Rob J Hyndan Working Paper 11/00
2 An iproved ethod for bandwidth selection when estiating ROC curves Peter G. Hall 1 and Rob J. Hyndan 1, 13 Septeber 00 Abstract: The receiver operating characteristic (ROC curve is used to describe the perforance of a diagnostic test which classifies observations into two groups. We introduce a new ethod for selecting bandwidths when coputing kernel estiates of ROC curves. Our technique allows for interaction between the distributions of each group of observations and gives substantial iproveent in MISE over other proposed ethods, especially when the two distributions are very different. Key words: Bandwidth selection; binary classification; kernel estiator; ROC curve. JEL classification: C1, C13, C14. 1 INTRODUCTION A receiver operating characteristic (ROC curve can be used to describe the perforance of a diagnostic test which classifies individuals into either group G 1 or group G. For exaple, G 1 ay contain individuals with a disease and G those without the disease. We assue that the diagnostic test is based on a continuous easureent T and that a person is classified as G 1 if T δ and G otherwise. Let G(t = Pr(T t G 1 and F (t = Pr(T t G denote the distribution functions of T for each group. (Thus F is the specificity of the test and 1 G is the sensitivity of the test. Then the ROC curve is defined as R(p = 1 G(F 1 (1 p where 0 p 1. Let {X 1,..., X } and {Y 1,..., Y n } denote independent saples of independent data fro G 1 and G, and let Fˆ and Ĝ denote their epirical distribution functions. Then a siple estiator of R(p is Rˆ(p = 1 Ĝ(Fˆ 1 (1 p, although this has the obvious weakness of being a step function while R(p is sooth. Zou, W.J. Hall & Shapiro (1997 and Lloyd (1998 proposed a sooth kernel estiator of R(p as follows. Let K(x be a continuous density function and L(x = x K(u du. The kernel estiators of F and G are F (t = 1 ( t L X i h 1 and G (t = i=1 i=1 1 ( t L 1 Centre for Matheatics and its Applications, Australian National University, Canberra ACT 000, Australia. Departent of Econoetrics and Business Statistics, Monash University, VIC 3800, Australia. Corresponding author: Rob Hyndan (Rob.Hyndan@onash.edu.au. 1 h Y i.
3 For the sake of siplicity we have used the sae kernel for each distribution, although of course this is not strictly necessary. The kernel estiator of R(p is then R (p = 1 G (F 1 (1 p. Qiu & Le (001 and Peng & Zhou (00 have discussed estiators alternative to R (p. Lloyd and Yong (1999 were the first to suggest epirical ethods for choosing bandwidths h 1 and h of appropriate size for R (p, but they treated the proble as one of estiating F and G separately, rather than of estiating the ROC function R. We shall show that by adopting the latter approach one can significantly reduce the surplus of ean squared error over its theoretically iniu level. This is particularly true in the practically interesting case where F and G are quite different. In the present paper we introduce and describe a bandwidth choice ethod which achieves these levels of perforance. A related proble, which leads to bandwidths of the correct order but without the correct constants, is that of soothing in distribution estiation. See, for exaple, Mielniczuk, Sarda and Vieu (1989, Sarda (1993, Altan and Legér (1995, and Bowan, Hall and Prvan (1998. METHODOLOGY.1 Optiality criterion and optial bandwidths If the tails of the distribution F are uch lighter than those of G then the error of an estiator of F in its tail can produce a relatively large contribution to the error of the corresponding estiator of G(F 1. As a result, if the L perforance criterion γ 1 (S = S [ E Ĝ(F 1 (p G(F (p] 1 dp (.1 for a set S [0, 1], is not weighted in an appropriate way then choice of the optial bandwidth in ters of γ 1 (S can be driven by relative tail properties of f and g. Forula (A.1 in the appendix will provide a theoretical illustration of this phenoenon. We suggest that the weight be chosen equal to f(f 1, so that the L criterion becoes γ(s = S [ E Ĝ(F 1 (p G(F (p] 1 f(f 1 (p dp. (. We shall show in the appendix that for this definition of ean integrated squared error, { } γ(s β(s E[F (t F (t] g (t + E[ Ĝ(t G(t] f (t dt (.3 F 1 (S where F 1 (S denotes the set of points F 1 (p with p S. Note particularly that the right-hand side is additive in the ean squared errors E( F F and E( Ĝ G, so that in principle h 1 and h ay be chosen individually, rather than together. That is, if h 1 and Hall and Hyndan: 13 Septeber 00
4 h iniise β 1 (S = E[F (t F (t] g (t dt and β (S = E[ Ĝ(t G(t] f (t dt, F 1 (S F 1 (S respectively, then they provide asyptotic iniisation of γ(s. To express optiality we take F 1 (S equal to the whole real line, obtaining the global criterion β(h 1, h = β 1 (h 1, h + β (h 1, h where β 1 (h 1, h = E[F (t F (t] g (t dt and β (h 1, h = E[Ĝ(t G(t] f (t dt β 1 = 1 (1 F F g + δ 1 + o( 1 h 1 + h 4 1 (.4 Suppose K is a copactly supported and syetric probability density, and f is bounded, continuous and square-integrable. Then arguents siilar to those of Azzalini (1981 show that E(F F = 1 [(1 F F h 1 ρ f] + ( 1 ρ h 1 f + o(n 1 h 1 + h 1 4, where ρ = (1 L(uL(u du, ρ = u K(u du. Of course, an analogous forula holds for E( Ĝ G, and so the forulae at (.4 adit siple asyptotic approxiations: β = n 1 (1 G G f + δ + o(n 1 h + h 4 where δ 1 = 1 h 1 ρ f g ρ h4 1 (f g (.5 and δ = n 1 h ρ f 1 g + 4 ρ h 4 (fg (.6 The asyptotically optial bandwidths are therefore h 1 = 1/3 c(f, g and h = n 1/3 c(g, f where { }/{ } c(f, g 3 = ρ f(u g (u du ρ [f (u g(u] du. A conventional plug-in rule for choosing h 1 and h ay be developed directly fro these forulae. However, it requires selection of pilot bandwidths for estiating f, g and their derivatives. The technique suggested in the next section avoids that difficulty. Hall and Hyndan: 13 Septeber 00 3
5 . Epirical choice of bandwidth Let f and ĝ denote leave-one-out kernel estiators of f and g, respectively: f ( x Xi1 ( x Xi (x h 1 = K K ( 1h 1 h 1 i 1 <i 1 h 1 ĝ ( y Yi1 ( y Yi (y h = K K. n(n 1h h 1 i 1 <i n h Let fˆ i(x h 1 = {( 1 h 1 } 1 j=i K{(x X j /h 1 }, and define ĝ i (y h analogously, and let f 1 and ĝ 1 denote the kernel estiators of (f and (g, respectively: f 1 (x h 1 = ĝ 1 (y h = ( 1h 4 1 n(n 1h 4 i 1 =1 i =1 n n K ( x X i1 K ( x X i h 1 h 1 ( K y Y i1 h i 1 =1 i =1 K ( y Y i. h Note that the latter two estiators include all ters whereas the other estiators are leave-one-out estiators. We include the diagonal ters in the estiators of (f and (g as they act like ridge paraeters and produce better epirical perforance. Now let (h 1, h = 1 h 1 ρ 1 ĝ 1 4 (X i h + 4 ρ h 1 n 1 f 1 (Y i h 1 ĝ i (Y i h i=1 i=1 n n 1 h ρ n 1 f 1 (Y i h ρ 4 h 1 ĝ 1 (X i h fˆ i(x i h 1. i=1 i=1 We could choose h 1 and h to iniize (h 1, h. To otivate this approach, note that E{ (h 1, h } = 1 h 1 ρ (Eĝ 1 f + 4 ρ 4 h 1 (Efˆ (Eĝ g n 1 h ρ (Efˆ g ρ h 4 (Eĝ (Efˆ f, (.7 which indicates that is an alost-unbiased approxiation to δ = δ 1 + δ ; copare (.7 with the su of the ters at (.5 and (.6. The relative size of stochastic error ay also be shown to be asyptotically negligible. Indeed, if n as n, if K is copactly supported and has a Hölder-continuous derivative, and if f and g are copactly supported and have three bounded derivatives, then (h 1, h /δ(h 1, h converges to 1 with probability 1, uniforly in n 1+ɛ h 1, h n ɛ for each 0 < τ < 1, as n. However, iniizing (h 1, h leads to soe nuerical instability. Instead, we constrain the iniization so that h 1 = ρh where ρ = h 1 /h and h 1 and h are the bandwidths selected for estiating F and G using the plug-in rule proposed by Lloyd and Yong (1999. Miniizing (h 1, h under this constraint provides values of h 1 and h which are suitable for estiating R (p. n Hall and Hyndan: 13 Septeber 00 4
6 3 SOME SIMULATIONS We copare the estiates obtained with our bandwidth selection ethod outlined above to those obtained by Lloyd and Yong (1999 using their plug-in rule. Let [ W (p = E G (F 1 (p G(F (p] 1 f(f 1 (p (3.1 denote ean squared error. Thus, ean integrated squared error, introduced at (., is given by γ(s = S W (p dp. The ideal but practically unattainable iniu of W (p, for a nonrando bandwidth, can be deduced by siulation, and will be denoted by W 0 (p. This value will be copared with its analogue, W 1 (p, obtained fro (3.1 using the values of h 1 and h chosen using the ethod outlined in Section.; and with W (p, obtained fro (3.1 using the values of h 1 and h chosen using the plug-in procedure suggested by Lloyd and Yong (1995. In our first exaple, illustrated in the first panel of Figure 1, we used Lloyd and Yong s (1999 odel, where F and G are N(0, 1 and N(1, 1 respectively. In the second exaple we chose F and G to be ore different; F was N(0, 1 and G was an equal ixture of N(, 1 and N(, 1. In both cases our ethod offers an iproveent, which as expected is greater when the distributions are further apart. The areas under the curves represent the increase in γ(s due to bandwidth selection. In these ters our ethod iproves on that of Lloyd and Yong (1999 by 1.% and 8.6%, in the respective exaples. Exaple 1 Exaple W i (x W 0 (x W i (x W 0 (x p p Figure 1: Solid lines: W 1 (p W 0 (p. Dashed lines: W (p W 0 (p. Hall and Hyndan: 13 Septeber 00 5
7 APPENDIX: Derivation of (.3 Assue that f and g have continuous derivatives and are bounded away fro 0 on S. Put A = F F, B = Ĝ G and C = F 1 F 1, and write I for the identity function. Then by Taylor expansion, I = F (F 1 + C = I + A(F 1 + C f(f 1 + o p ( A(F 1 + C, whence it follows that C = [A(F 1 /f(f 1 ] + o p ( A(F 1. Hence, g(f 1 Ĝ(F 1 G(F 1 = B(F 1 f(f 1 A(F 1 + o p ( A(F 1 + B(F 1. (A.1 Note the ratio g(f 1 /f(f 1 on the right-hand side of (A.1. Since the variance of A equals (1 F F then the unweighted criterion γ 1, defined at (.1, can be largely deterined by the value of (g/f (1 F F in the tails if this quantity is not bounded. Using instead the weighted criterion γ, defined at (., we ay deduce fro (A.1, related coputations and the independence of the saples that S E[ Ĝ(F 1 G(F 1 ] f(f 1 = [1 + o(1] which is equivalent to (.3. F 1 (S [E(B f + E(A g ] REFERENCES Altan, N. and Léger, C. (1995. Bandwidth selection for kernel distribution function estiation. J. Statist. Plann. Inf. 46, Azzalini, A. (1981. A note on the estiation of a distribution function and quantiles by a kernel ethod. Bioetrika 68, Bowan, A.W., Hall, P. and Prvan, T. (1998. Cross-validation for the soothing of distribution functions. Bioetrika 85, Lloyd, C.J. (1998. The use of soothed ROC curves to suarise and copare diagnostic systes. J. Aer. Statist. Assoc. 93, Lloyd, C.J. and Yong, Z (1999. Kernel estiators of the ROC curve are better than epirical. Statist. Prob. Letters 44, 1 8. Mielniczuk, J., Sarda, P. and Vieu, P. (1989. Local data-driven bandwidth choice for density estiation. J. Statist. Plann. Inf. 3, Peng, L. and Zhou, X.-H. (00. Local linear soothing of receiver operator characteristic (ROC curves. J. Statist. Plann. Inf., to appear. Qiu, P. and Le, C. (001. ROC curve estiation based on local soothing. J. Statist. Coput. and Siul. 70, Sarda, P. (1993. Soothing paraeter selection for sooth distribution functions. J. Statist. Plann. Inf. 35, Zou, K.H., Hall, W.J. and Shapiro, D.E. (1997. Sooth non-paraetric receiver operating characteristic (ROC curves for continuous diagnostic tests. Statistics in Medicine Hall and Hyndan: 13 Septeber 00 6
Nonparametric confidence intervals. for receiver operating characteristic curves
Nonparametric confidence intervals for receiver operating characteristic curves Peter G. Hall 1, Rob J. Hyndman 2, and Yanan Fan 3 5 December 2003 Abstract: We study methods for constructing confidence
More informationA Simple Regression Problem
A Siple Regression Proble R. M. Castro March 23, 2 In this brief note a siple regression proble will be introduced, illustrating clearly the bias-variance tradeoff. Let Y i f(x i ) + W i, i,..., n, where
More informationNon-Parametric Non-Line-of-Sight Identification 1
Non-Paraetric Non-Line-of-Sight Identification Sinan Gezici, Hisashi Kobayashi and H. Vincent Poor Departent of Electrical Engineering School of Engineering and Applied Science Princeton University, Princeton,
More informationModel Fitting. CURM Background Material, Fall 2014 Dr. Doreen De Leon
Model Fitting CURM Background Material, Fall 014 Dr. Doreen De Leon 1 Introduction Given a set of data points, we often want to fit a selected odel or type to the data (e.g., we suspect an exponential
More informationSMOOTH ESTIMATION OF ROC CURVE IN THE PRESENCE OF AUXILIARY INFORMATION
J Syst Sci Coplex 4: 99 944 SMOOTH ESTIMATION OF ROC CURVE IN THE PRESENCE OF AUXILIARY INFORMATION Yong ZHOU Haibo ZHOU Yunbei MA DOI:.7/s44--795-7 Received: May 7 / Revised: August 8 c The Editorial
More informationTesting equality of variances for multiple univariate normal populations
University of Wollongong Research Online Centre for Statistical & Survey Methodology Working Paper Series Faculty of Engineering and Inforation Sciences 0 esting equality of variances for ultiple univariate
More informationBoosting with log-loss
Boosting with log-loss Marco Cusuano-Towner Septeber 2, 202 The proble Suppose we have data exaples {x i, y i ) i =... } for a two-class proble with y i {, }. Let F x) be the predictor function with the
More informationAN EFFICIENT CLASS OF CHAIN ESTIMATORS OF POPULATION VARIANCE UNDER SUB-SAMPLING SCHEME
J. Japan Statist. Soc. Vol. 35 No. 005 73 86 AN EFFICIENT CLASS OF CHAIN ESTIMATORS OF POPULATION VARIANCE UNDER SUB-SAMPLING SCHEME H. S. Jhajj*, M. K. Shara* and Lovleen Kuar Grover** For estiating the
More informationKeywords: Estimator, Bias, Mean-squared error, normality, generalized Pareto distribution
Testing approxiate norality of an estiator using the estiated MSE and bias with an application to the shape paraeter of the generalized Pareto distribution J. Martin van Zyl Abstract In this work the norality
More informationMachine Learning Basics: Estimators, Bias and Variance
Machine Learning Basics: Estiators, Bias and Variance Sargur N. srihari@cedar.buffalo.edu This is part of lecture slides on Deep Learning: http://www.cedar.buffalo.edu/~srihari/cse676 1 Topics in Basics
More informationBiostatistics Department Technical Report
Biostatistics Departent Technical Report BST006-00 Estiation of Prevalence by Pool Screening With Equal Sized Pools and a egative Binoial Sapling Model Charles R. Katholi, Ph.D. Eeritus Professor Departent
More informationAn Approximate Model for the Theoretical Prediction of the Velocity Increase in the Intermediate Ballistics Period
An Approxiate Model for the Theoretical Prediction of the Velocity... 77 Central European Journal of Energetic Materials, 205, 2(), 77-88 ISSN 2353-843 An Approxiate Model for the Theoretical Prediction
More informationA MESHSIZE BOOSTING ALGORITHM IN KERNEL DENSITY ESTIMATION
A eshsize boosting algorith in kernel density estiation A MESHSIZE BOOSTING ALGORITHM IN KERNEL DENSITY ESTIMATION C.C. Ishiekwene, S.M. Ogbonwan and J.E. Osewenkhae Departent of Matheatics, University
More informationBootstrapping Dependent Data
Bootstrapping Dependent Data One of the key issues confronting bootstrap resapling approxiations is how to deal with dependent data. Consider a sequence fx t g n t= of dependent rando variables. Clearly
More informationGEE ESTIMATORS IN MIXTURE MODEL WITH VARYING CONCENTRATIONS
ACTA UIVERSITATIS LODZIESIS FOLIA OECOOMICA 3(3142015 http://dx.doi.org/10.18778/0208-6018.314.03 Olesii Doronin *, Rostislav Maiboroda ** GEE ESTIMATORS I MIXTURE MODEL WITH VARYIG COCETRATIOS Abstract.
More informationBlock designs and statistics
Bloc designs and statistics Notes for Math 447 May 3, 2011 The ain paraeters of a bloc design are nuber of varieties v, bloc size, nuber of blocs b. A design is built on a set of v eleents. Each eleent
More informationC na (1) a=l. c = CO + Clm + CZ TWO-STAGE SAMPLE DESIGN WITH SMALL CLUSTERS. 1. Introduction
TWO-STGE SMPLE DESIGN WITH SMLL CLUSTERS Robert G. Clark and David G. Steel School of Matheatics and pplied Statistics, University of Wollongong, NSW 5 ustralia. (robert.clark@abs.gov.au) Key Words: saple
More informationTail Estimation of the Spectral Density under Fixed-Domain Asymptotics
Tail Estiation of the Spectral Density under Fixed-Doain Asyptotics Wei-Ying Wu, Chae Young Li and Yiin Xiao Wei-Ying Wu, Departent of Statistics & Probability Michigan State University, East Lansing,
More informationNecessity of low effective dimension
Necessity of low effective diension Art B. Owen Stanford University October 2002, Orig: July 2002 Abstract Practitioners have long noticed that quasi-monte Carlo ethods work very well on functions that
More informationEnsemble Based on Data Envelopment Analysis
Enseble Based on Data Envelopent Analysis So Young Sohn & Hong Choi Departent of Coputer Science & Industrial Systes Engineering, Yonsei University, Seoul, Korea Tel) 82-2-223-404, Fax) 82-2- 364-7807
More informationBayes Decision Rule and Naïve Bayes Classifier
Bayes Decision Rule and Naïve Bayes Classifier Le Song Machine Learning I CSE 6740, Fall 2013 Gaussian Mixture odel A density odel p(x) ay be ulti-odal: odel it as a ixture of uni-odal distributions (e.g.
More informationEstimation of the Mean of the Exponential Distribution Using Maximum Ranked Set Sampling with Unequal Samples
Open Journal of Statistics, 4, 4, 64-649 Published Online Septeber 4 in SciRes http//wwwscirporg/ournal/os http//ddoiorg/436/os4486 Estiation of the Mean of the Eponential Distribution Using Maiu Ranked
More informationStochastic Subgradient Methods
Stochastic Subgradient Methods Lingjie Weng Yutian Chen Bren School of Inforation and Coputer Science University of California, Irvine {wengl, yutianc}@ics.uci.edu Abstract Stochastic subgradient ethods
More informationIntroduction to Machine Learning. Recitation 11
Introduction to Machine Learning Lecturer: Regev Schweiger Recitation Fall Seester Scribe: Regev Schweiger. Kernel Ridge Regression We now take on the task of kernel-izing ridge regression. Let x,...,
More informationSupport Vector Machine Classification of Uncertain and Imbalanced data using Robust Optimization
Recent Researches in Coputer Science Support Vector Machine Classification of Uncertain and Ibalanced data using Robust Optiization RAGHAV PAT, THEODORE B. TRAFALIS, KASH BARKER School of Industrial Engineering
More informationThe Distribution of the Covariance Matrix for a Subset of Elliptical Distributions with Extension to Two Kurtosis Parameters
journal of ultivariate analysis 58, 96106 (1996) article no. 0041 The Distribution of the Covariance Matrix for a Subset of Elliptical Distributions with Extension to Two Kurtosis Paraeters H. S. Steyn
More informationAn l 1 Regularized Method for Numerical Differentiation Using Empirical Eigenfunctions
Journal of Matheatical Research with Applications Jul., 207, Vol. 37, No. 4, pp. 496 504 DOI:0.3770/j.issn:2095-265.207.04.0 Http://jre.dlut.edu.cn An l Regularized Method for Nuerical Differentiation
More informationComparison of Stability of Selected Numerical Methods for Solving Stiff Semi- Linear Differential Equations
International Journal of Applied Science and Technology Vol. 7, No. 3, Septeber 217 Coparison of Stability of Selected Nuerical Methods for Solving Stiff Sei- Linear Differential Equations Kwaku Darkwah
More informationOn Constant Power Water-filling
On Constant Power Water-filling Wei Yu and John M. Cioffi Electrical Engineering Departent Stanford University, Stanford, CA94305, U.S.A. eails: {weiyu,cioffi}@stanford.edu Abstract This paper derives
More informationComputable Shell Decomposition Bounds
Coputable Shell Decoposition Bounds John Langford TTI-Chicago jcl@cs.cu.edu David McAllester TTI-Chicago dac@autoreason.co Editor: Leslie Pack Kaelbling and David Cohn Abstract Haussler, Kearns, Seung
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Notes for EE7C (Spring 018: Convex Optiization and Approxiation Instructor: Moritz Hardt Eail: hardt+ee7c@berkeley.edu Graduate Instructor: Max Sichowitz Eail: sichow+ee7c@berkeley.edu October 15,
More informationA Note on the Applied Use of MDL Approximations
A Note on the Applied Use of MDL Approxiations Daniel J. Navarro Departent of Psychology Ohio State University Abstract An applied proble is discussed in which two nested psychological odels of retention
More informationESTIMATING AND FORMING CONFIDENCE INTERVALS FOR EXTREMA OF RANDOM POLYNOMIALS. A Thesis. Presented to. The Faculty of the Department of Mathematics
ESTIMATING AND FORMING CONFIDENCE INTERVALS FOR EXTREMA OF RANDOM POLYNOMIALS A Thesis Presented to The Faculty of the Departent of Matheatics San Jose State University In Partial Fulfillent of the Requireents
More informationA Simplified Analytical Approach for Efficiency Evaluation of the Weaving Machines with Automatic Filling Repair
Proceedings of the 6th SEAS International Conference on Siulation, Modelling and Optiization, Lisbon, Portugal, Septeber -4, 006 0 A Siplified Analytical Approach for Efficiency Evaluation of the eaving
More informationProc. of the IEEE/OES Seventh Working Conference on Current Measurement Technology UNCERTAINTIES IN SEASONDE CURRENT VELOCITIES
Proc. of the IEEE/OES Seventh Working Conference on Current Measureent Technology UNCERTAINTIES IN SEASONDE CURRENT VELOCITIES Belinda Lipa Codar Ocean Sensors 15 La Sandra Way, Portola Valley, CA 98 blipa@pogo.co
More informatione-companion ONLY AVAILABLE IN ELECTRONIC FORM
OPERATIONS RESEARCH doi 10.1287/opre.1070.0427ec pp. ec1 ec5 e-copanion ONLY AVAILABLE IN ELECTRONIC FORM infors 07 INFORMS Electronic Copanion A Learning Approach for Interactive Marketing to a Custoer
More informationCombining Classifiers
Cobining Classifiers Generic ethods of generating and cobining ultiple classifiers Bagging Boosting References: Duda, Hart & Stork, pg 475-480. Hastie, Tibsharini, Friedan, pg 246-256 and Chapter 10. http://www.boosting.org/
More informationComputational and Statistical Learning Theory
Coputational and Statistical Learning Theory Proble sets 5 and 6 Due: Noveber th Please send your solutions to learning-subissions@ttic.edu Notations/Definitions Recall the definition of saple based Radeacher
More informationE0 370 Statistical Learning Theory Lecture 6 (Aug 30, 2011) Margin Analysis
E0 370 tatistical Learning Theory Lecture 6 (Aug 30, 20) Margin Analysis Lecturer: hivani Agarwal cribe: Narasihan R Introduction In the last few lectures we have seen how to obtain high confidence bounds
More informationA remark on a success rate model for DPA and CPA
A reark on a success rate odel for DPA and CPA A. Wieers, BSI Version 0.5 andreas.wieers@bsi.bund.de Septeber 5, 2018 Abstract The success rate is the ost coon evaluation etric for easuring the perforance
More informationLogLog-Beta and More: A New Algorithm for Cardinality Estimation Based on LogLog Counting
LogLog-Beta and More: A New Algorith for Cardinality Estiation Based on LogLog Counting Jason Qin, Denys Ki, Yuei Tung The AOLP Core Data Service, AOL, 22000 AOL Way Dulles, VA 20163 E-ail: jasonqin@teaaolco
More informationMeta-Analytic Interval Estimation for Bivariate Correlations
Psychological Methods 2008, Vol. 13, No. 3, 173 181 Copyright 2008 by the Aerican Psychological Association 1082-989X/08/$12.00 DOI: 10.1037/a0012868 Meta-Analytic Interval Estiation for Bivariate Correlations
More informationAN OPTIMAL SHRINKAGE FACTOR IN PREDICTION OF ORDERED RANDOM EFFECTS
Statistica Sinica 6 016, 1709-178 doi:http://dx.doi.org/10.5705/ss.0014.0034 AN OPTIMAL SHRINKAGE FACTOR IN PREDICTION OF ORDERED RANDOM EFFECTS Nilabja Guha 1, Anindya Roy, Yaakov Malinovsky and Gauri
More informationSharp Time Data Tradeoffs for Linear Inverse Problems
Sharp Tie Data Tradeoffs for Linear Inverse Probles Saet Oyak Benjain Recht Mahdi Soltanolkotabi January 016 Abstract In this paper we characterize sharp tie-data tradeoffs for optiization probles used
More informationCOS 424: Interacting with Data. Written Exercises
COS 424: Interacting with Data Hoework #4 Spring 2007 Regression Due: Wednesday, April 18 Written Exercises See the course website for iportant inforation about collaboration and late policies, as well
More informationCh 12: Variations on Backpropagation
Ch 2: Variations on Backpropagation The basic backpropagation algorith is too slow for ost practical applications. It ay take days or weeks of coputer tie. We deonstrate why the backpropagation algorith
More informationPattern Recognition and Machine Learning. Learning and Evaluation for Pattern Recognition
Pattern Recognition and Machine Learning Jaes L. Crowley ENSIMAG 3 - MMIS Fall Seester 2017 Lesson 1 4 October 2017 Outline Learning and Evaluation for Pattern Recognition Notation...2 1. The Pattern Recognition
More informationAnalyzing Simulation Results
Analyzing Siulation Results Dr. John Mellor-Cruey Departent of Coputer Science Rice University johnc@cs.rice.edu COMP 528 Lecture 20 31 March 2005 Topics for Today Model verification Model validation Transient
More informationExtension of CSRSM for the Parametric Study of the Face Stability of Pressurized Tunnels
Extension of CSRSM for the Paraetric Study of the Face Stability of Pressurized Tunnels Guilhe Mollon 1, Daniel Dias 2, and Abdul-Haid Soubra 3, M.ASCE 1 LGCIE, INSA Lyon, Université de Lyon, Doaine scientifique
More informationThe Methods of Solution for Constrained Nonlinear Programming
Research Inventy: International Journal Of Engineering And Science Vol.4, Issue 3(March 2014), PP 01-06 Issn (e): 2278-4721, Issn (p):2319-6483, www.researchinventy.co The Methods of Solution for Constrained
More informationShannon Sampling II. Connections to Learning Theory
Shannon Sapling II Connections to Learning heory Steve Sale oyota echnological Institute at Chicago 147 East 60th Street, Chicago, IL 60637, USA E-ail: sale@athberkeleyedu Ding-Xuan Zhou Departent of Matheatics,
More informationOptimal Jackknife for Discrete Time and Continuous Time Unit Root Models
Optial Jackknife for Discrete Tie and Continuous Tie Unit Root Models Ye Chen and Jun Yu Singapore Manageent University January 6, Abstract Maxiu likelihood estiation of the persistence paraeter in the
More informationConstructing Locally Best Invariant Tests of the Linear Regression Model Using the Density Function of a Maximal Invariant
Aerican Journal of Matheatics and Statistics 03, 3(): 45-5 DOI: 0.593/j.ajs.03030.07 Constructing Locally Best Invariant Tests of the Linear Regression Model Using the Density Function of a Maxial Invariant
More informationComputational and Statistical Learning Theory
Coputational and Statistical Learning Theory TTIC 31120 Prof. Nati Srebro Lecture 2: PAC Learning and VC Theory I Fro Adversarial Online to Statistical Three reasons to ove fro worst-case deterinistic
More informationIntelligent Systems: Reasoning and Recognition. Perceptrons and Support Vector Machines
Intelligent Systes: Reasoning and Recognition Jaes L. Crowley osig 1 Winter Seester 2018 Lesson 6 27 February 2018 Outline Perceptrons and Support Vector achines Notation...2 Linear odels...3 Lines, Planes
More informationResearch in Area of Longevity of Sylphon Scraies
IOP Conference Series: Earth and Environental Science PAPER OPEN ACCESS Research in Area of Longevity of Sylphon Scraies To cite this article: Natalia Y Golovina and Svetlana Y Krivosheeva 2018 IOP Conf.
More informationTEST OF HOMOGENEITY OF PARALLEL SAMPLES FROM LOGNORMAL POPULATIONS WITH UNEQUAL VARIANCES
TEST OF HOMOGENEITY OF PARALLEL SAMPLES FROM LOGNORMAL POPULATIONS WITH UNEQUAL VARIANCES S. E. Ahed, R. J. Tokins and A. I. Volodin Departent of Matheatics and Statistics University of Regina Regina,
More informationPh 20.3 Numerical Solution of Ordinary Differential Equations
Ph 20.3 Nuerical Solution of Ordinary Differential Equations Due: Week 5 -v20170314- This Assignent So far, your assignents have tried to failiarize you with the hardware and software in the Physics Coputing
More informationEFFICIENT GEOMETRIC METHODS FOR KERNEL DENSITY ESTIMATION BASED INDEPENDENT COMPONENT ANALYSIS
15th European Signal Processing Conference (EUSIPCO 2007), Poznan, Poland, Septeber 3-7, 2007, copyright by EURASIP EFFICIENT GEOMETRIC METHODS FOR KERNEL DENSITY ESTIMATION BASED INDEPENDENT COMPONENT
More informationExperimental Design For Model Discrimination And Precise Parameter Estimation In WDS Analysis
City University of New York (CUNY) CUNY Acadeic Works International Conference on Hydroinforatics 8-1-2014 Experiental Design For Model Discriination And Precise Paraeter Estiation In WDS Analysis Giovanna
More informationThe proofs of Theorem 1-3 are along the lines of Wied and Galeano (2013).
A Appendix: Proofs The proofs of Theore 1-3 are along the lines of Wied and Galeano (2013) Proof of Theore 1 Let D[d 1, d 2 ] be the space of càdlàg functions on the interval [d 1, d 2 ] equipped with
More informationPseudo-marginal Metropolis-Hastings: a simple explanation and (partial) review of theory
Pseudo-arginal Metropolis-Hastings: a siple explanation and (partial) review of theory Chris Sherlock Motivation Iagine a stochastic process V which arises fro soe distribution with density p(v θ ). Iagine
More informationOn the approximation of Feynman-Kac path integrals
On the approxiation of Feynan-Kac path integrals Stephen D. Bond, Brian B. Laird, and Benedict J. Leikuhler University of California, San Diego, Departents of Matheatics and Cheistry, La Jolla, CA 993,
More informationComputable Shell Decomposition Bounds
Journal of Machine Learning Research 5 (2004) 529-547 Subitted 1/03; Revised 8/03; Published 5/04 Coputable Shell Decoposition Bounds John Langford David McAllester Toyota Technology Institute at Chicago
More informationNonmonotonic Networks. a. IRST, I Povo (Trento) Italy, b. Univ. of Trento, Physics Dept., I Povo (Trento) Italy
Storage Capacity and Dynaics of Nononotonic Networks Bruno Crespi a and Ignazio Lazzizzera b a. IRST, I-38050 Povo (Trento) Italy, b. Univ. of Trento, Physics Dept., I-38050 Povo (Trento) Italy INFN Gruppo
More informationBest Linear Unbiased and Invariant Reconstructors for the Past Records
BULLETIN of the MALAYSIAN MATHEMATICAL SCIENCES SOCIETY http:/athusy/bulletin Bull Malays Math Sci Soc (2) 37(4) (2014), 1017 1028 Best Linear Unbiased and Invariant Reconstructors for the Past Records
More informationŞtefan ŞTEFĂNESCU * is the minimum global value for the function h (x)
7Applying Nelder Mead s Optiization Algorith APPLYING NELDER MEAD S OPTIMIZATION ALGORITHM FOR MULTIPLE GLOBAL MINIMA Abstract Ştefan ŞTEFĂNESCU * The iterative deterinistic optiization ethod could not
More informationOBJECTIVES INTRODUCTION
M7 Chapter 3 Section 1 OBJECTIVES Suarize data using easures of central tendency, such as the ean, edian, ode, and idrange. Describe data using the easures of variation, such as the range, variance, and
More informationarxiv: v1 [cs.ds] 3 Feb 2014
arxiv:40.043v [cs.ds] 3 Feb 04 A Bound on the Expected Optiality of Rando Feasible Solutions to Cobinatorial Optiization Probles Evan A. Sultani The Johns Hopins University APL evan@sultani.co http://www.sultani.co/
More informationMSEC MODELING OF DEGRADATION PROCESSES TO OBTAIN AN OPTIMAL SOLUTION FOR MAINTENANCE AND PERFORMANCE
Proceeding of the ASME 9 International Manufacturing Science and Engineering Conference MSEC9 October 4-7, 9, West Lafayette, Indiana, USA MSEC9-8466 MODELING OF DEGRADATION PROCESSES TO OBTAIN AN OPTIMAL
More informationPORTMANTEAU TESTS FOR ARMA MODELS WITH INFINITE VARIANCE
doi:10.1111/j.1467-9892.2007.00572.x PORTMANTEAU TESTS FOR ARMA MODELS WITH INFINITE VARIANCE By J.-W. Lin and A. I. McLeod The University of Western Ontario First Version received Septeber 2006 Abstract.
More informationLeast Squares Fitting of Data
Least Squares Fitting of Data David Eberly, Geoetric Tools, Redond WA 98052 https://www.geoetrictools.co/ This work is licensed under the Creative Coons Attribution 4.0 International License. To view a
More informationOn Conditions for Linearity of Optimal Estimation
On Conditions for Linearity of Optial Estiation Erah Akyol, Kuar Viswanatha and Kenneth Rose {eakyol, kuar, rose}@ece.ucsb.edu Departent of Electrical and Coputer Engineering University of California at
More informationThe Transactional Nature of Quantum Information
The Transactional Nature of Quantu Inforation Subhash Kak Departent of Coputer Science Oklahoa State University Stillwater, OK 7478 ABSTRACT Inforation, in its counications sense, is a transactional property.
More informationNonlinear Log-Periodogram Regression for Perturbed Fractional Processes
Nonlinear Log-Periodogra Regression for Perturbed Fractional Processes Yixiao Sun Departent of Econoics Yale University Peter C. B. Phillips Cowles Foundation for Research in Econoics Yale University First
More informationChapter 2. Small-Signal Model Parameter Extraction Method
Chapter Sall-Signal Model Paraeter Extraction Method In this chapter, we introduce a new paraeter extraction technique for sall-signal HBT odeling. Figure - shows the sall-signal equivalent circuit of
More informationA model reduction approach to numerical inversion for a parabolic partial differential equation
Inverse Probles Inverse Probles 30 (204) 250 (33pp) doi:0.088/0266-56/30/2/250 A odel reduction approach to nuerical inversion for a parabolic partial differential equation Liliana Borcea, Vladiir Drusin
More informationOn a few Iterative Methods for Solving Nonlinear Equations
On a few Iterative Methods for Solving Nonlinear Equations Gyurhan Nedzhibov Laboratory of Matheatical Modelling, Shuen University, Shuen 971, Bulgaria e-ail: gyurhan@shu-bg.net Abstract In this study
More informationStatistics and Probability Letters
Statistics and Probability Letters 79 2009 223 233 Contents lists available at ScienceDirect Statistics and Probability Letters journal hoepage: www.elsevier.co/locate/stapro A CLT for a one-diensional
More informationDistributed Subgradient Methods for Multi-agent Optimization
1 Distributed Subgradient Methods for Multi-agent Optiization Angelia Nedić and Asuan Ozdaglar October 29, 2007 Abstract We study a distributed coputation odel for optiizing a su of convex objective functions
More informationarxiv: v1 [math.st] 23 Feb 2012
he Annals of Statistics 20, Vol. 39, No. 5, 2330 2355 DOI: 0.24/-AOS898 c Institute of Matheatical Statistics, 20 arxiv:202.534v [ath.s] 23 Feb 202 OPIMAL ESIMAION OF HE MEAN FUNCION BASED ON DISCREELY
More informationA Comparative Study of Parametric and Nonparametric Regressions
Iranian Econoic Review, Vol.16, No.30, Fall 011 A Coparative Study of Paraetric and Nonparaetric Regressions Shahra Fattahi Received: 010/08/8 Accepted: 011/04/4 Abstract his paper evaluates inflation
More informationIn this chapter, we consider several graph-theoretic and probabilistic models
THREE ONE GRAPH-THEORETIC AND STATISTICAL MODELS 3.1 INTRODUCTION In this chapter, we consider several graph-theoretic and probabilistic odels for a social network, which we do under different assuptions
More informationare equal to zero, where, q = p 1. For each gene j, the pairwise null and alternative hypotheses are,
Page of 8 Suppleentary Materials: A ultiple testing procedure for ulti-diensional pairwise coparisons with application to gene expression studies Anjana Grandhi, Wenge Guo, Shyaal D. Peddada S Notations
More informationA new type of lower bound for the largest eigenvalue of a symmetric matrix
Linear Algebra and its Applications 47 7 9 9 www.elsevier.co/locate/laa A new type of lower bound for the largest eigenvalue of a syetric atrix Piet Van Mieghe Delft University of Technology, P.O. Box
More informationA Smoothed Boosting Algorithm Using Probabilistic Output Codes
A Soothed Boosting Algorith Using Probabilistic Output Codes Rong Jin rongjin@cse.su.edu Dept. of Coputer Science and Engineering, Michigan State University, MI 48824, USA Jian Zhang jian.zhang@cs.cu.edu
More informationProbability Distributions
Probability Distributions In Chapter, we ephasized the central role played by probability theory in the solution of pattern recognition probles. We turn now to an exploration of soe particular exaples
More informationInspection; structural health monitoring; reliability; Bayesian analysis; updating; decision analysis; value of information
Cite as: Straub D. (2014). Value of inforation analysis with structural reliability ethods. Structural Safety, 49: 75-86. Value of Inforation Analysis with Structural Reliability Methods Daniel Straub
More informationCHAPTER 19: Single-Loop IMC Control
When I coplete this chapter, I want to be able to do the following. Recognize that other feedback algoriths are possible Understand the IMC structure and how it provides the essential control features
More informationNuclear Instruments and Methods in Physics Research B 262 (2007)
Nuclear Instruents and Methods in Physics Research B 262 (27) 33 322 NIM B Bea Interactions with Materials & Atos www.elsevier.co/locate/ni Peak shape ethods for general order theroluinescence glow-peaks:
More informationWarning System of Dangerous Chemical Gas in Factory Based on Wireless Sensor Network
565 A publication of CHEMICAL ENGINEERING TRANSACTIONS VOL. 59, 07 Guest Editors: Zhuo Yang, Junie Ba, Jing Pan Copyright 07, AIDIC Servizi S.r.l. ISBN 978-88-95608-49-5; ISSN 83-96 The Italian Association
More informationResearch Article Approximate Multidegree Reduction of λ-bézier Curves
Matheatical Probles in Engineering Volue 6 Article ID 87 pages http://dxdoiorg//6/87 Research Article Approxiate Multidegree Reduction of λ-bézier Curves Gang Hu Huanxin Cao and Suxia Zhang Departent of
More informationIN modern society that various systems have become more
Developent of Reliability Function in -Coponent Standby Redundant Syste with Priority Based on Maxiu Entropy Principle Ryosuke Hirata, Ikuo Arizono, Ryosuke Toohiro, Satoshi Oigawa, and Yasuhiko Takeoto
More informationHyperbolic Horn Helical Mass Spectrometer (3HMS) James G. Hagerman Hagerman Technology LLC & Pacific Environmental Technologies April 2005
Hyperbolic Horn Helical Mass Spectroeter (3HMS) Jaes G Hageran Hageran Technology LLC & Pacific Environental Technologies April 5 ABSTRACT This paper describes a new type of ass filter based on the REFIMS
More informationThe degree of a typical vertex in generalized random intersection graph models
Discrete Matheatics 306 006 15 165 www.elsevier.co/locate/disc The degree of a typical vertex in generalized rando intersection graph odels Jerzy Jaworski a, Michał Karoński a, Dudley Stark b a Departent
More information1 Identical Parallel Machines
FB3: Matheatik/Inforatik Dr. Syaantak Das Winter 2017/18 Optiizing under Uncertainty Lecture Notes 3: Scheduling to Miniize Makespan In any standard scheduling proble, we are given a set of jobs J = {j
More informationInternational Journal of Pure and Applied Mathematics Volume 37 No , IMPROVED DATA DRIVEN CONTROL CHARTS
International Journal of Pure and Applied Matheatics Volue 37 No. 3 2007, 423-438 IMPROVED DATA DRIVEN CONTROL CHARTS Wille Albers 1, Wilbert C.M. Kallenberg 2 1,2 Departent of Applied Matheatics Faculty
More informationComparing Probabilistic Forecasting Systems with the Brier Score
1076 W E A T H E R A N D F O R E C A S T I N G VOLUME 22 Coparing Probabilistic Forecasting Systes with the Brier Score CHRISTOPHER A. T. FERRO School of Engineering, Coputing and Matheatics, University
More informationA note on the multiplication of sparse matrices
Cent. Eur. J. Cop. Sci. 41) 2014 1-11 DOI: 10.2478/s13537-014-0201-x Central European Journal of Coputer Science A note on the ultiplication of sparse atrices Research Article Keivan Borna 12, Sohrab Aboozarkhani
More informationAn Improved Particle Filter with Applications in Ballistic Target Tracking
Sensors & ransducers Vol. 72 Issue 6 June 204 pp. 96-20 Sensors & ransducers 204 by IFSA Publishing S. L. http://www.sensorsportal.co An Iproved Particle Filter with Applications in Ballistic arget racing
More information