Sensitivity and Asymptotic Error Theory
|
|
- Sybil Jennings
- 5 years ago
- Views:
Transcription
1 Sensitivity and Asymptotic Error Theory H.T. Banks and Marie Davidian MA-ST 810 Fall, 2009 North Carolina State University Raleigh, NC Center for Quantitative Sciences in Biomedicine North Carolina State University 1
2 Introduction Goal: To illustrate use and possible pitfalls in statistical inverse problem analysis of sensitivity equations and related standard errors. When data is sampled from a region near a steady state of a curve, the optimized parameters will not necessarily improve. The standard errors for estimated parameters may increase with more data points, contrary to intuition. To explain this, we will look at a specific example. 2
3 Statistical Review We assume that n scalar longitudinal observations are represented by the statistical model Y j f j (θ 0 ) + E j, j = 1, 2,..., n. Thus we seek to use data {y j } for the observation process {Y j } with the model to seek a value ˆθ n that minimizes n J n (θ) = f j (θ) y j 2. j=1 3
4 Since Y j is a random variable, we have that the estimator ˆθ n OLS is also a random variable with a distribution called the sampling distribution. For large n, the sampling distribution approximately satisfies ˆθ n OLS(Y ) N p (θ 0, σ 2 0[χ T (θ 0 )χ(θ 0 )] 1 ) := N p (θ 0, Σ 0 ), where χ(θ) = F θ (θ) is the n p sensitivity matrix with elements χ jk (θ) = f j(θ) θ k and F θ (θ) (f 1θ (θ),..., f nθ (θ)) T. F θ (θ) can be found using difference quotients, sensitivity equations, or direct analytic calculations (in some cases). 4
5 Typically several ways to compute the matrix F θ (which are actually the well known sensitivity functions widely used in applied mathematics and engineering see the discussions in [BDSS]). First, the elements of the matrix χ = (χ jk ) can always be estimated using the forward difference χ jk (θ) = f(t j, θ) θ k f(t j, θ + h k ) f(t j, θ), h k where h k is a p-vector with a nonzero entry in only the k th component. But, of course, the choice of h k can be problematic in practice. 5
6 Alternatively, if the f(t j, θ) correspond to longitudinal observations y(t j ) = Cx(t j ; θ) of solutions x R N to a parameterized N-vector differential equation system x = g(t, x(t), θ), then one can use the N p matrix sensitivity equations (see [BEG, BN] and the references therein) to obtain d dt ( ) x θ f(t j, θ) θ k = g x x θ + g θ = C x(t j, θ) θ k. In some cases the function f(t j, θ) may be sufficiently simple so as to allow one to derive analytical expressions for the components of F θ. But for more complex models often encountered, can use automatic differentiation software packages. (1) 6
7 Since θ 0, σ 0 are not known, we follow standard practice and use the approximation Σ 0 Σ(ˆθ n ) = ˆσ 2 [χ T (ˆθ n )χ(ˆθ n )] 1 (2) where ˆθ n is the parameter estimate obtained, and the approximation ˆσ 2 to σ0 2 is given by σ0 2 ˆσ 2 = 1 n f j (ˆθ n ) y j 2. (3) n p Standard errors to be used in confidence interval calculations are thus given by SE k (ˆθ n ) = Σ kk (ˆθ n ), k = 1, 2,..., p and consequently we define the confidence level parameters associated with the estimated parameters so that j=1 P {ˆθ n k t 1 α/2 SE k (ˆθ n ) < θ n k < ˆθ n k + t 1 α/2 SE k (ˆθ n )} = 1 α. 7
8 Under reasonable assumptions on smoothness and regularity (the smoothness requirements for model solutions are readily verified using continuous dependence results for differential equations in most examples; the regularity requirements include, among others, conditions on how the observations are taken as sample size increases, i.e., as n ), the standard nonlinear regression approximation theory ([DG, G, J], and Chapter 12 of [SeWi]) for asymptotic (as n ) distributions can be invoked. When one is taking longitudinal samples corresponding to solutions of a dynamical system, the n p sensitivity matrix depends explicitly on where in time the observations are taken when f(t j, θ) = Cx(t j, θ) as mentioned above. That is, the sensitivity matrix ( ) f(tj, θ) χ(θ) = F θ (θ) = θ k depends on the number n and the nature (for example, how 8
9 taken) of the sampling times {t j }. Moreover, it is the matrix [χ T χ] 1 in (2) and the parameter ˆσ 2 in (3) that ultimately determine the standard errors and confidence intervals. At first investigation of (3), it appears that an increased number n of samples might drive ˆσ 2 (and hence the SE) to zero as long as this is done in a way to maintain a bound on the residual sum of squares in (3). However, we observe that the condition number of the Fisher matrix F = χ T χ is also very important in these considerations and increasing the sampling could potentially adversely affect the inversion of χ T χ. In this regard, we note that among the important hypotheses in the asymptotic statistical theory (see p. 571 of [SeWi]) is the existence of a matrix function Ω(θ) such that 1 n χnt (θ)χ n (θ) Ω(θ) uniformly in θ as n, with Ω 0 = Ω(θ 0 ) a nonsingular matrix. 9
10 Moreover, there are conditions on the observation times {t j } such as existence of a finite measure μ on [0, T ] satisfying 1 n Σn j=1h(t j ) for each continuous function h. T 0 h(t)dμ(t) (4) It is these conditions that are rather easily violated in practice when one is dealing with data from differential equation systems, especially near an equilibrium or steady state (see the examples of [BEG]). 10
11 Example Consider the logistic growth population model dx (1 dt = rx x ) K where K is the carrying capacity and r is the intrinsic growth rate. The solution is given by x K x(t) = 1 + K ( ). K x 0 1 e rt K/2 11
12 Analytical Considerations We will examine the problem x = ax bx 2, where as k = a x 0 x(t) = = a/b 1 + ( a/b x 0 1)e at a b + ke at b. Here x(t) has an asymptote at a b = K. 12
13 We begin with an ordinary least squares problem for θ = (a, b, x 0 ) where X = x θ = with the covariance matrix x(t 1 ) a. x(t n ) a Σ = ˆσ 2 (X T X) 1 = 1 n 3 x(t 1 ) b. x(t n ) b x(t 1 ) x 0. x(t n ) x 0 n y j f(t j, ˆθ n ) 2 (X T X) 1 j=1 and the estimate of the standard error SE k = ˆσ 2 (X T X) 1 kk, k = 1, 2, 3. 13
14 Let R 0 be the region where t [0, τ 1 ], R 1 be the region corresponding to t [τ 1, τ 2 ], and R 2 where t [τ 2, ). a/b x 0 R 1 R 2 R 0 τ 1 τ 2 Figure 1: Partition of solution curve into distinct regions. 14
15 In order to divide the problem x = ax bx 2 into separate regions, we consider the solution x = a b + ke at and then examine the partial derivatives x a, x b, and x x 0. As t, which corresponds to region R 2, we see that x(t) a b. Also, x(t) We will denote x(0) = x 0 = a b+k as t 0, which corresponds to region R 0. a b+k. 15
16 We have the following partial derivatives x a x b = x x 0 = b + (atk b)e at (b + ke at ) 2, = a(1 e at ) (b + ke at ) 2, a 2 e at x 2 0 (b + ke at ) 2, which are n 1 vectors for t = t 1,..., t n. The matrix X is composed of these vectors and we obtain the following matrix, X T X = n j=1 ( x(tj ) a x(t j ) b ) 2 x(t j ) x(t j ) x(t j ) x(t j ) a b a x ( ) 0 2 x(tj ) x(t j ) x(t j ) b b x 0 x(t j ) a x(t j ) x(t j ) x 0 a x(t j ) x(t j ) x 0 b ( x(tj ) x 0 ) 2. 16
17 If we sample data from R 0, where t j < τ 1 for j = 1... n, we have x(t j ) a Also notice that 0, n j=1 x(t j ) b 0, x(t j ) x 0 x(t j ) x 0 = n 1 = n j=1 x(t j ) x 0 1. for t j < τ 1 in R 0. Hence, in this region, X T X n 17
18 Next, consider region R 2, where t j > τ 2 for j = 1... n. Notice that in R 2, and hence, X T X x(t j ) a n j=1 x(t j ) a x(t j ) b 1 b, x(t j ) b x(t j ) a x(t j ) a a b 2, x(t j ) x 0 0, x(t j ) b 0 x(t j ) x(t j ) x(t j ) a b b 0 = n b a 2 b 0 3 b a b 3 a 2 Notice that the second column of X T Xis a scalar multiple of the first column, which means that we should be able to estimate the ratio a b but not the individual parameters. 18
19 Implementation We create a simulated data set, y j, j = 1,..., n, using the analytical solution with a specific θ 0. We want to solve an inverse problem The cost function uses ode15s to approximate the solution and returns J(θ) = n y j f(t j ; θ) 2 j=1 where f(t; θ) = x(t; a, b, x 0 ) is the approximation to the solution. We use the MATLAB function fminsearch to optimize θ in order to obtain the minimized cost J(ˆθ n ). Here, ˆθ n represents the optimized value of θ over n data points. 19
20 80 70 Steep Moderate Flat Exact solutions on the whole region Solution Time Figure 2: Three sets of simulated data: a relatively flat curve with θ 0 = (0.5, 0.1, 0.1), a moderately sloped curve with θ 0 = (0.7, 0.04, 0.1), and a steep curve with θ 0 = (0.8, 0.01, 0.1). Define R 0 [0, 2], R 1 [2, 12], R 2 [12, 16]. 20
21 Results Regional Partitions Regions R 0, R 1, R 2 Standard Errors Regions R 0 R 1 and R 1 R 2 Simulated Noise 21
22 Region R 0 The flat curve with β 0 =( 0.6, 0.2, 0.5). 13 The moderate curve with β 0 =( 0.8, 0.1, 0.3). 13 The steep curve with β 0 =( 0.7, 0.2, 0.5) Model Model Model Time Time Time Figure 3: Simulated data plotted with the solution obtained from the estimated parameters in region R 0 for the a) flat curve with θ 0 = (0.6, 0.2, 0.5) b) moderate curve with θ 0 = (0.8, 0.1, 0.3) c) steep curve with θ 0 = (0.7, 0.2, 0.5). 22
23 Region R 1 The flat curve with β 0 =( 0.6, 0.2, 0.5). 61 The moderate curve with β 0 =( 0.8, 0.1, 0.3). 61 The steep curve with β 0 =( 0.7, 0.2, 0.5) Model Model Model Time Time Time Figure 4: Simulated data plotted with the solution obtained from the estimated parameters in region R 1 for the a) flat curve with θ 0 = (0.6, 0.2, 0.5) b) moderate curve with θ 0 = (0.8, 0.1, 0.3) c) steep curve with θ 0 = (0.7, 0.2, 0.5). 23
24 Region R 2 The flat curve with β 0 =( 0.6, 0.2, 0.5). 25 The moderate curve with β 0 =( 0.8, 0.1, 0.3). 25 The steep curve with β 0 =( 0.7, 0.2, 0.5) Model Model Model Time Time Time Figure 5: Simulated data plotted with the solution obtained from the estimated parameters in region R 2 for the a) flat curve with θ 0 = (0.6, 0.2, 0.5) b) moderate curve with θ 0 = (0.8, 0.1, 0.3) c) steep curve with θ 0 = (0.7, 0.2, 0.5). 24
25 Regions R 0 and R 1 The moderate curve with θ 0 = (0.7, 0.04, 0.1) using θ 0 = (0.8, 0.1, 0.3). n 0 n 1 ˆθ n Standard Errors 25 0 (0.7244,0.1414,0.0992) (0.0049, , ) 49 0 (0.7241,0.1401,0.0992) (0.0094, , ) (0.6998,0.0397,0.1) (2.2059e 4, e 5, e 4 ) Table 1: The standard errors for a, b, and x 0 for the optimized ˆθ n. 25
26 Regions R 1 and R 2 The moderate curve with θ 0 = (0.7, 0.04, 0.1) using θ 0 = (0.8, 0.1, 0.3). n 1 n 2 ˆθ n Standard Errors 0 25 (1.0057,0.0582,0.3249) (1.0794, , ) 0 49 (1.0058,0.0582,0.3249) (2.1739, , ) (0.6996,0.04,0.1003) (5.0686e 3, e 4, e 3 ) Table 2: The standard errors for a, b, and x 0 for the optimized ˆθ n. 26
27 Simulated Noise We consider two sets of simulated noisy data for the moderate curve. One set corresponds to R 0 and is perturbed using σ 0 = and the other corresponds to R 2 and has σ 0 = 0.5. These values represent an added noise level of approximately ten percent at the lowest bound in each region. 20 σ 0 =0 σ 0 = Time Figure 6: Noisy data using σ 0 = 0.5 where θ 0 = (0.8, 0.1, 0.3). 27
28 Regions R 0 and R 1 with Noise The moderate curve with θ 0 = (0.7, 0.04, 0.1) using θ 0 = (0.8, 0.1, 0.3). n 0 n 1 ˆθn Standard Errors 25 0 (0.7244,0.1455,0.0988) (0.052, , ) 49 0 (0.7245,0.1452,0.0988) (0.1060, , ) (0.7086,0.0517,0.0988) (0.0497, , 0.033) (0.703,0.0435,0.0994) (0.091, , ) 0 25 (0.7058,0.0491,0.0994) (0.0133, , 0.009) 0 49 (0.6958,0.0362,0.1008) (0.0252, , ) Table 3: The standard errors for the optimized ˆθ n with σ = using data sampled from the interval [0, 4]. 28
29 Regions R 1 and R 2 with Noise The moderate curve with θ 0 = (0.7, 0.04, 0.1) using θ 0 = (0.8, 0.1, 0.3). n 1 n 2 ˆθn Standard Errors 0 25 (1.0058,0.0582,0.3249) (2.7953, , ) 0 49 (1.0057,0.0582,0.3249) (6.3606, , ) (0.5286,0.0297,0.4397) (3.7015, , ) (0.673,0.0384,0.123) ( , 0.679, ) 25 0 (0.5175,0.0281,0.4307) (1.0073, , ) 49 0 (0.4713,0.0249,0.5921) (2.1167, , ) Table 4: The standard errors for the optimized ˆθ n with σ = 0.5 using data sampled from the interval [8, 16]. 29
30 Conclusions Sampling data from regions near steady states alone may not provide sufficient information to obtain all parameters with any degree of accuracy (lack of sensitivity). Including additional data from the same regions near steady states will only increase the standard errors without significant improvement to the optimized parameters (no new information from new sample points). When extra data points are sampled in a way that provides new information from a region where the solution is sensitive to the parameters, then the parameter estimation will be improved (i.e., region R 1 from this problem). 30
31 References [BDSS] H.T. Banks, M. Davidian, J.R. Samuels, Jr., and K.L. Sutton, An Inverse Problem Statistical Methodology Summary, CRSC-TR08-01, January, 2008; Chapter 11 in Statistical Estimation Approaches in Epidemiology, (edited by Gerardo Chowell, Mac Hyman, Nick Hengartner, Luis M.A Bettencourt and Carlos Castillo-Chavez), Springer, Berlin Heidelberg New York, 2009, pp [BN] H. T. Banks and H. K. Nguyen, Sensitivity of dynamical system to Banach space parameters, CRSC Tech Rep., CRSC-TR05-13, N.C. State University, February, 2005; J. Math Anal. Appl., 323 (2006), [BEG] H.T. Banks, Stacey L. Ernstberger and Sarah L. Grove, Standard errors and confidence intervals in inverse problems: Sensitivity and associated pitfalls, CRSC-TR06-10, March, 2006; J. Inverse and Ill-posed Problems, 15 (2007), [CB] G. Casella and R. L. Berger, Statistical Inference, Duxbury, California, [DG] M. Davidian and D. Giltinan, Nonlinear Models for Repeated Measurement, Chapman & Hall, London, [G] A. R. Gallant, Nonlinear Statistical Models, John Wiley & Sons, Inc., New York, [J] R. I. Jennrich, Asymptotic properties of non-linear least squares estimators., Ann. Math. Statist., 40: ,
32 [K] M. Kot, Elements of Mathematical Ecology, Cambridge University Press, 2001, p [SeWi] G. A. F. Seber and C. J. Wild, Nonlinear Regression, John Wiley & Sons, Inc., New York,
33 Region R 1 with σ 0 = The moderate curve with θ 0 = (0.7, 0.04, 0.1) using θ 0 = (0.8, 0.1, 0.3). n 1 Interval ˆθn Standard Errors 25 [2,4] (0.7058,0.0491,0.0994) (0.0133, , 0.009) 49 [2,4] (0.6958,0.0362,0.1008) (0.0252, , ) 49 [2,6] (0.6981,0.0394,0.1006) (0.0355, , ) 73 [2,8] (0.6999,0.04,0.1001) (0.0794, , ) Table 5: The standard errors for the optimized ˆθ n with σ = using data sampled from various intervals within R 1. 33
34 Region R 1 with σ 0 = 0.5 The moderate curve with θ 0 = (0.7, 0.04, 0.1) using θ 0 = (0.8, 0.1, 0.3). n 1 Interval ˆθn Standard Errors 25 [8,12] (0.5175,0.0281,0.4307) (1.0073, , ) 49 [8,12] (0.4713,0.0249,0.5921) (2.1167, , ) 49 [4,12] (0.6720,0.0383,0.1254) (5.1571, , ) Table 6: The standard errors for the optimized ˆθ n with σ = 0.5 using data sampled from various intervals within R 1. 34
Statistical Error Model Comparison for Logistic Growth of Green Algae (Raphidocelis subcapiala)
Statistical Error Model Comparison for Logistic Growth of Green Algae (Raphidocelis subcapiala) H.T. Banks, Elizabeth Collins, Kevin Flores, Prayag Pershad, Michael Stemkovski, Lyric Stephenson Center
More informationAn Inverse Problem Statistical Methodology Summary
An Inverse Problem Statistical Methodology Summary H.T. Banks, M. Davidian, J.R. Samuels, Jr. and Karyn L.Sutton Center for Research in Scientific Computation and Center for Quantitative Sciences in Biomedicine
More informationSensitivity Functions and Their Uses in Inverse Problems
Sensitivity Functions and Their Uses in Inverse Problems H.T. Banks 1, Sava Dediu 2 and Stacey L. Ernstberger 3 Center for Research in Scientific Computation North Carolina State University Raleigh, NC
More informationThe Estimation of the Effective Reproductive Number from Disease. Outbreak Data
The Estimation of the Effective Reproductive Number from Disease Outbreak Data Ariel Cintrón-Arias 1,5 Carlos Castillo-Chávez 2, Luís M. A. Bettencourt 3, Alun L. Lloyd 4,5, and H. T. Banks 4,5 1 Statistical
More informationBounded Error Parameter Estimation for Models Described by Ordinary and Delay Differential Equations
Bounded Error Parameter Estimation for Models Described by Ordinary and Delay Differential Equations John A. Burns and Adam F. Childers Interdisciplinary Center for Applied Mathematics Virginia Tech Blacksburg,
More informationParameter Estimation for Random Differential Equation Models
Parameter Estimation for Random Differential Equation Models H.T. Banks and Michele L. Joyner Center for Research in Scientific Computation North Carolina State University Raleigh, NC, United States and
More informationMath Ordinary Differential Equations
Math 411 - Ordinary Differential Equations Review Notes - 1 1 - Basic Theory A first order ordinary differential equation has the form x = f(t, x) (11) Here x = dx/dt Given an initial data x(t 0 ) = x
More informationLecture 9. Systems of Two First Order Linear ODEs
Math 245 - Mathematics of Physics and Engineering I Lecture 9. Systems of Two First Order Linear ODEs January 30, 2012 Konstantin Zuev (USC) Math 245, Lecture 9 January 30, 2012 1 / 15 Agenda General Form
More informationMarcia Gumpertz and Sastry G. Pantula Department of Statistics North Carolina State University Raleigh, NC
A Simple Approach to Inference in Random Coefficient Models March 8, 1988 Marcia Gumpertz and Sastry G. Pantula Department of Statistics North Carolina State University Raleigh, NC 27695-8203 Key Words
More informationLarge Sample Properties of Estimators in the Classical Linear Regression Model
Large Sample Properties of Estimators in the Classical Linear Regression Model 7 October 004 A. Statement of the classical linear regression model The classical linear regression model can be written in
More informationThermal Damage Detection and Characterization in Porous Materials
Thermal Damage Detection and Characterization in Porous Materials H.T. Banks and Amanda Keck Criner Center for Research in Scientific Computation North Carolina State University Raleigh, NC 27695-8212
More informationStochastic Optimization with Inequality Constraints Using Simultaneous Perturbations and Penalty Functions
International Journal of Control Vol. 00, No. 00, January 2007, 1 10 Stochastic Optimization with Inequality Constraints Using Simultaneous Perturbations and Penalty Functions I-JENG WANG and JAMES C.
More informationUncertainty Quantification for Inverse Problems. November 7, 2011
Uncertainty Quantification for Inverse Problems November 7, 2011 Outline UQ and inverse problems Review: least-squares Review: Gaussian Bayesian linear model Parametric reductions for IP Bias, variance
More informationA Likelihood Ratio Test
A Likelihood Ratio Test David Allen University of Kentucky February 23, 2012 1 Introduction Earlier presentations gave a procedure for finding an estimate and its standard error of a single linear combination
More informationTo Estimate or Not to Estimate?
To Estimate or Not to Estimate? Benjamin Kedem and Shihua Wen In linear regression there are examples where some of the coefficients are known but are estimated anyway for various reasons not least of
More informationAkaike Information Criterion
Akaike Information Criterion Shuhua Hu Center for Research in Scientific Computation North Carolina State University Raleigh, NC February 7, 2012-1- background Background Model statistical model: Y j =
More informationParameter Estimation and Uncertainty Quantification for an Epidemic Model
Parameter Estimation and Uncertainty Quantification for an Epidemic Model A. Capaldi Center for Quantitative Sciences in Biomedicine and Department of Mathematics North Carolina State University Raleigh,
More informationChapter 4: Constrained estimators and tests in the multiple linear regression model (Part III)
Chapter 4: Constrained estimators and tests in the multiple linear regression model (Part III) Florian Pelgrin HEC September-December 2010 Florian Pelgrin (HEC) Constrained estimators September-December
More informationLOCAL FIXED POINT THEORY INVOLVING THREE OPERATORS IN BANACH ALGEBRAS. B. C. Dhage. 1. Introduction
Topological Methods in Nonlinear Analysis Journal of the Juliusz Schauder Center Volume 24, 24, 377 386 LOCAL FIXED POINT THEORY INVOLVING THREE OPERATORS IN BANACH ALGEBRAS B. C. Dhage Abstract. The present
More informationAnalysis of the AIC Statistic for Optimal Detection of Small Changes in Dynamic Systems
Analysis of the AIC Statistic for Optimal Detection of Small Changes in Dynamic Systems Jeremy S. Conner and Dale E. Seborg Department of Chemical Engineering University of California, Santa Barbara, CA
More informationChapter 1: Introduction
Chapter 1: Introduction Definition: A differential equation is an equation involving the derivative of a function. If the function depends on a single variable, then only ordinary derivatives appear and
More informationThe Bayesian Approach to Multi-equation Econometric Model Estimation
Journal of Statistical and Econometric Methods, vol.3, no.1, 2014, 85-96 ISSN: 2241-0384 (print), 2241-0376 (online) Scienpress Ltd, 2014 The Bayesian Approach to Multi-equation Econometric Model Estimation
More informationAuthor's Copy. A sensitivity matrix based methodology for inverse problem formulation
c de Gruyter 2009 J. Inv. Ill-Posed Problems 17 (2009), 545 564 DOI 10.1515 / JIIP.2009.034 A sensitivity matrix based methodology for inverse problem formulation A. Cintrón-Arias, H. T. Banks, A. Capaldi
More informationCovariance function estimation in Gaussian process regression
Covariance function estimation in Gaussian process regression François Bachoc Department of Statistics and Operations Research, University of Vienna WU Research Seminar - May 2015 François Bachoc Gaussian
More informationSTAT 730 Chapter 4: Estimation
STAT 730 Chapter 4: Estimation Timothy Hanson Department of Statistics, University of South Carolina Stat 730: Multivariate Analysis 1 / 23 The likelihood We have iid data, at least initially. Each datum
More informationA Robust Controller for Scalar Autonomous Optimal Control Problems
A Robust Controller for Scalar Autonomous Optimal Control Problems S. H. Lam 1 Department of Mechanical and Aerospace Engineering Princeton University, Princeton, NJ 08544 lam@princeton.edu Abstract Is
More informationRegularization Parameter Estimation for Least Squares: A Newton method using the χ 2 -distribution
Regularization Parameter Estimation for Least Squares: A Newton method using the χ 2 -distribution Rosemary Renaut, Jodi Mead Arizona State and Boise State September 2007 Renaut and Mead (ASU/Boise) Scalar
More informationComputationally Efficient Estimation of Multilevel High-Dimensional Latent Variable Models
Computationally Efficient Estimation of Multilevel High-Dimensional Latent Variable Models Tihomir Asparouhov 1, Bengt Muthen 2 Muthen & Muthen 1 UCLA 2 Abstract Multilevel analysis often leads to modeling
More informationGaussian Estimation under Attack Uncertainty
Gaussian Estimation under Attack Uncertainty Tara Javidi Yonatan Kaspi Himanshu Tyagi Abstract We consider the estimation of a standard Gaussian random variable under an observation attack where an adversary
More informationNONSTANDARD NUMERICAL METHODS FOR A CLASS OF PREDATOR-PREY MODELS WITH PREDATOR INTERFERENCE
Sixth Mississippi State Conference on Differential Equations and Computational Simulations, Electronic Journal of Differential Equations, Conference 15 (2007), pp. 67 75. ISSN: 1072-6691. URL: http://ejde.math.txstate.edu
More informationAsymptotic Relative Efficiency in Estimation
Asymptotic Relative Efficiency in Estimation Robert Serfling University of Texas at Dallas October 2009 Prepared for forthcoming INTERNATIONAL ENCYCLOPEDIA OF STATISTICAL SCIENCES, to be published by Springer
More informationBIBO STABILITY AND ASYMPTOTIC STABILITY
BIBO STABILITY AND ASYMPTOTIC STABILITY FRANCESCO NORI Abstract. In this report with discuss the concepts of bounded-input boundedoutput stability (BIBO) and of Lyapunov stability. Examples are given to
More informationOSCILLATION AND ASYMPTOTIC STABILITY OF A DELAY DIFFERENTIAL EQUATION WITH RICHARD S NONLINEARITY
2004 Conference on Diff. Eqns. and Appl. in Math. Biology, Nanaimo, BC, Canada. Electronic Journal of Differential Equations, Conference 12, 2005, pp. 21 27. ISSN: 1072-6691. URL: http://ejde.math.txstate.edu
More informationCOMPARISON OF GMM WITH SECOND-ORDER LEAST SQUARES ESTIMATION IN NONLINEAR MODELS. Abstract
Far East J. Theo. Stat. 0() (006), 179-196 COMPARISON OF GMM WITH SECOND-ORDER LEAST SQUARES ESTIMATION IN NONLINEAR MODELS Department of Statistics University of Manitoba Winnipeg, Manitoba, Canada R3T
More information2tdt 1 y = t2 + C y = which implies C = 1 and the solution is y = 1
Lectures - Week 11 General First Order ODEs & Numerical Methods for IVPs In general, nonlinear problems are much more difficult to solve than linear ones. Unfortunately many phenomena exhibit nonlinear
More informationSome New Aspects of Dose-Response Models with Applications to Multistage Models Having Parameters on the Boundary
Some New Aspects of Dose-Response Models with Applications to Multistage Models Having Parameters on the Boundary Bimal Sinha Department of Mathematics & Statistics University of Maryland, Baltimore County,
More informationThe Unifying View on Ordinary Differential Equations and Automatic Differentiation, yet with a Gap to Fill Alexander Gofen.
The Unifying View on Ordinary Differential Equations and Automatic Differentiation, yet with a Gap to Fill Alexander Gofen Conventions Differentiation is understood as for holomorphic functions in the
More informationSize and Shape of Confidence Regions from Extended Empirical Likelihood Tests
Biometrika (2014),,, pp. 1 13 C 2014 Biometrika Trust Printed in Great Britain Size and Shape of Confidence Regions from Extended Empirical Likelihood Tests BY M. ZHOU Department of Statistics, University
More informationBias-free Sparse Regression with Guaranteed Consistency
Bias-free Sparse Regression with Guaranteed Consistency Wotao Yin (UCLA Math) joint with: Stanley Osher, Ming Yan (UCLA) Feng Ruan, Jiechao Xiong, Yuan Yao (Peking U) UC Riverside, STATS Department March
More informationLecture 2: From Linear Regression to Kalman Filter and Beyond
Lecture 2: From Linear Regression to Kalman Filter and Beyond January 18, 2017 Contents 1 Batch and Recursive Estimation 2 Towards Bayesian Filtering 3 Kalman Filter and Bayesian Filtering and Smoothing
More informationON THE ASYMPTOTIC STABILITY IN TERMS OF TWO MEASURES FOR FUNCTIONAL DIFFERENTIAL EQUATIONS. G. Makay
ON THE ASYMPTOTIC STABILITY IN TERMS OF TWO MEASURES FOR FUNCTIONAL DIFFERENTIAL EQUATIONS G. Makay Student in Mathematics, University of Szeged, Szeged, H-6726, Hungary Key words and phrases: Lyapunov
More informationWhy ZMPE When You Can MUPE?
T E C O L O T E R ESE ARC H, I NC. Bridging Engineering and Economics Since 1973 Why ZMPE When You Can MUPE? 6 th Joint ISPA/SCEA Conference Dr. Shu-Ping Hu and Alfred Smith 12-15 June 2007 Los Angeles
More informationInteraction effects for continuous predictors in regression modeling
Interaction effects for continuous predictors in regression modeling Testing for interactions The linear regression model is undoubtedly the most commonly-used statistical model, and has the advantage
More informationEECE Adaptive Control
EECE 574 - Adaptive Control Basics of System Identification Guy Dumont Department of Electrical and Computer Engineering University of British Columbia January 2010 Guy Dumont (UBC) EECE574 - Basics of
More informationIntroduction to Initial Value Problems
Chapter 2 Introduction to Initial Value Problems The purpose of this chapter is to study the simplest numerical methods for approximating the solution to a first order initial value problem (IVP). Because
More informationOptimization Problems
Optimization Problems The goal in an optimization problem is to find the point at which the minimum (or maximum) of a real, scalar function f occurs and, usually, to find the value of the function at that
More informationDiscussion of the paper Inference for Semiparametric Models: Some Questions and an Answer by Bickel and Kwon
Discussion of the paper Inference for Semiparametric Models: Some Questions and an Answer by Bickel and Kwon Jianqing Fan Department of Statistics Chinese University of Hong Kong AND Department of Statistics
More informationEXISTENCE AND UNIQUENESS OF SOLUTIONS FOR FOURTH-ORDER BOUNDARY-VALUE PROBLEMS IN BANACH SPACES
Electronic Journal of Differential Equations, Vol. 9(9), No. 33, pp. 1 8. ISSN: 17-6691. URL: http://ejde.math.txstate.edu or http://ejde.math.unt.edu ftp ejde.math.txstate.edu EXISTENCE AND UNIQUENESS
More informationHigh-dimensional statistics and data analysis Course Part I
and data analysis Course Part I 3 - Computation of p-values in high-dimensional regression Jérémie Bigot Institut de Mathématiques de Bordeaux - Université de Bordeaux Master MAS-MSS, Université de Bordeaux,
More informationCS 195-5: Machine Learning Problem Set 1
CS 95-5: Machine Learning Problem Set Douglas Lanman dlanman@brown.edu 7 September Regression Problem Show that the prediction errors y f(x; ŵ) are necessarily uncorrelated with any linear function of
More informationReview of Classical Least Squares. James L. Powell Department of Economics University of California, Berkeley
Review of Classical Least Squares James L. Powell Department of Economics University of California, Berkeley The Classical Linear Model The object of least squares regression methods is to model and estimate
More informationBayesian and frequentist inference
Bayesian and frequentist inference Nancy Reid March 26, 2007 Don Fraser, Ana-Maria Staicu Overview Methods of inference Asymptotic theory Approximate posteriors matching priors Examples Logistic regression
More informationUnderstanding Regressions with Observations Collected at High Frequency over Long Span
Understanding Regressions with Observations Collected at High Frequency over Long Span Yoosoon Chang Department of Economics, Indiana University Joon Y. Park Department of Economics, Indiana University
More informationIllustration of the Varying Coefficient Model for Analyses the Tree Growth from the Age and Space Perspectives
TR-No. 14-06, Hiroshima Statistical Research Group, 1 11 Illustration of the Varying Coefficient Model for Analyses the Tree Growth from the Age and Space Perspectives Mariko Yamamura 1, Keisuke Fukui
More informationσ(a) = a N (x; 0, 1 2 ) dx. σ(a) = Φ(a) =
Until now we have always worked with likelihoods and prior distributions that were conjugate to each other, allowing the computation of the posterior distribution to be done in closed form. Unfortunately,
More informationStatistical Measures of Uncertainty in Inverse Problems
Statistical Measures of Uncertainty in Inverse Problems Workshop on Uncertainty in Inverse Problems Institute for Mathematics and Its Applications Minneapolis, MN 19-26 April 2002 P.B. Stark Department
More informationApplicable Analysis and Discrete Mathematics available online at FRUSTRATED KURAMOTO MODEL GENERALISE EQUITABLE PARTITIONS
Applicable Analysis and Discrete Mathematics available online at http://pefmath.etf.rs Appl. Anal. Discrete Math. x (xxxx), xxx xxx. doi:10.98/aadmxxxxxxxx α-kuramoto PARTITIONS FROM THE FRUSTRATED KURAMOTO
More informationTolerance limits for a ratio of normal random variables
Tolerance limits for a ratio of normal random variables Lanju Zhang 1, Thomas Mathew 2, Harry Yang 1, K. Krishnamoorthy 3 and Iksung Cho 1 1 Department of Biostatistics MedImmune, Inc. One MedImmune Way,
More information1 Linear Regression and Correlation
Math 10B with Professor Stankova Worksheet, Discussion #27; Tuesday, 5/1/2018 GSI name: Roy Zhao 1 Linear Regression and Correlation 1.1 Concepts 1. Often when given data points, we want to find the line
More informationNonlinear Systems and Control Lecture # 19 Perturbed Systems & Input-to-State Stability
p. 1/1 Nonlinear Systems and Control Lecture # 19 Perturbed Systems & Input-to-State Stability p. 2/1 Perturbed Systems: Nonvanishing Perturbation Nominal System: Perturbed System: ẋ = f(x), f(0) = 0 ẋ
More informationEconometrics II - EXAM Answer each question in separate sheets in three hours
Econometrics II - EXAM Answer each question in separate sheets in three hours. Let u and u be jointly Gaussian and independent of z in all the equations. a Investigate the identification of the following
More informationNONTRIVIAL SOLUTIONS FOR SUPERQUADRATIC NONAUTONOMOUS PERIODIC SYSTEMS. Shouchuan Hu Nikolas S. Papageorgiou. 1. Introduction
Topological Methods in Nonlinear Analysis Journal of the Juliusz Schauder Center Volume 34, 29, 327 338 NONTRIVIAL SOLUTIONS FOR SUPERQUADRATIC NONAUTONOMOUS PERIODIC SYSTEMS Shouchuan Hu Nikolas S. Papageorgiou
More informationsimple if it completely specifies the density of x
3. Hypothesis Testing Pure significance tests Data x = (x 1,..., x n ) from f(x, θ) Hypothesis H 0 : restricts f(x, θ) Are the data consistent with H 0? H 0 is called the null hypothesis simple if it completely
More informationSt. Petersburg Math. J. 13 (2001),.1 Vol. 13 (2002), No. 4
St Petersburg Math J 13 (21), 1 Vol 13 (22), No 4 BROCKETT S PROBLEM IN THE THEORY OF STABILITY OF LINEAR DIFFERENTIAL EQUATIONS G A Leonov Abstract Algorithms for nonstationary linear stabilization are
More informationECON 5350 Class Notes Functional Form and Structural Change
ECON 5350 Class Notes Functional Form and Structural Change 1 Introduction Although OLS is considered a linear estimator, it does not mean that the relationship between Y and X needs to be linear. In this
More informationOn robust and efficient estimation of the center of. Symmetry.
On robust and efficient estimation of the center of symmetry Howard D. Bondell Department of Statistics, North Carolina State University Raleigh, NC 27695-8203, U.S.A (email: bondell@stat.ncsu.edu) Abstract
More informationA NEW PROOF OF THE WIENER HOPF FACTORIZATION VIA BASU S THEOREM
J. Appl. Prob. 49, 876 882 (2012 Printed in England Applied Probability Trust 2012 A NEW PROOF OF THE WIENER HOPF FACTORIZATION VIA BASU S THEOREM BRIAN FRALIX and COLIN GALLAGHER, Clemson University Abstract
More informationOutline of GLMs. Definitions
Outline of GLMs Definitions This is a short outline of GLM details, adapted from the book Nonparametric Regression and Generalized Linear Models, by Green and Silverman. The responses Y i have density
More informationEVANESCENT SOLUTIONS FOR LINEAR ORDINARY DIFFERENTIAL EQUATIONS
EVANESCENT SOLUTIONS FOR LINEAR ORDINARY DIFFERENTIAL EQUATIONS Cezar Avramescu Abstract The problem of existence of the solutions for ordinary differential equations vanishing at ± is considered. AMS
More informationFinal Exam November 24, Problem-1: Consider random walk with drift plus a linear time trend: ( t
Problem-1: Consider random walk with drift plus a linear time trend: y t = c + y t 1 + δ t + ϵ t, (1) where {ϵ t } is white noise with E[ϵ 2 t ] = σ 2 >, and y is a non-stochastic initial value. (a) Show
More informationMath 256: Applied Differential Equations: Final Review
Math 256: Applied Differential Equations: Final Review Chapter 1: Introduction, Sec 1.1, 1.2, 1.3 (a) Differential Equation, Mathematical Model (b) Direction (Slope) Field, Equilibrium Solution (c) Rate
More informationLogistic regression model for survival time analysis using time-varying coefficients
Logistic regression model for survival time analysis using time-varying coefficients Accepted in American Journal of Mathematical and Management Sciences, 2016 Kenichi SATOH ksatoh@hiroshima-u.ac.jp Research
More informationPrediction of ESTSP Competition Time Series by Unscented Kalman Filter and RTS Smoother
Prediction of ESTSP Competition Time Series by Unscented Kalman Filter and RTS Smoother Simo Särkkä, Aki Vehtari and Jouko Lampinen Helsinki University of Technology Department of Electrical and Communications
More informationTesting a Normal Covariance Matrix for Small Samples with Monotone Missing Data
Applied Mathematical Sciences, Vol 3, 009, no 54, 695-70 Testing a Normal Covariance Matrix for Small Samples with Monotone Missing Data Evelina Veleva Rousse University A Kanchev Department of Numerical
More informationLinear Inverse Problems. A MATLAB Tutorial Presented by Johnny Samuels
Linear Inverse Problems A MATLAB Tutorial Presented by Johnny Samuels What do we want to do? We want to develop a method to determine the best fit to a set of data: e.g. The Plan Review pertinent linear
More informationAsymptotic behaviour of solutions of third order nonlinear differential equations
Acta Univ. Sapientiae, Mathematica, 3, 2 (211) 197 211 Asymptotic behaviour of solutions of third order nonlinear differential equations A. T. Ademola Department of Mathematics University of Ibadan Ibadan,
More informationAPPROXIMATE SOLUTION OF A SYSTEM OF LINEAR EQUATIONS WITH RANDOM PERTURBATIONS
APPROXIMATE SOLUTION OF A SYSTEM OF LINEAR EQUATIONS WITH RANDOM PERTURBATIONS P. Date paresh.date@brunel.ac.uk Center for Analysis of Risk and Optimisation Modelling Applications, Department of Mathematical
More informationParameter Selection Methods in Inverse Problem Formulation
Parameter Selection Methods in Inverse Problem Formulation H. T. Banks and Ariel Cintrón-Arias,2 Center for Research in Scientific Computation Center for Quantitative Sciences in Biomedicine Department
More information(θ θ ), θ θ = 2 L(θ ) θ θ θ θ θ (θ )= H θθ (θ ) 1 d θ (θ )
Setting RHS to be zero, 0= (θ )+ 2 L(θ ) (θ θ ), θ θ = 2 L(θ ) 1 (θ )= H θθ (θ ) 1 d θ (θ ) O =0 θ 1 θ 3 θ 2 θ Figure 1: The Newton-Raphson Algorithm where H is the Hessian matrix, d θ is the derivative
More informationVariational Principal Components
Variational Principal Components Christopher M. Bishop Microsoft Research 7 J. J. Thomson Avenue, Cambridge, CB3 0FB, U.K. cmbishop@microsoft.com http://research.microsoft.com/ cmbishop In Proceedings
More informationBasic Analysis of Data
Basic Analysis of Data Department of Chemical Engineering Prof. Geoff Silcox Fall 008 1.0 Reporting the Uncertainty in a Measured Quantity At the request of your supervisor, you have ventured out into
More information36. Multisample U-statistics and jointly distributed U-statistics Lehmann 6.1
36. Multisample U-statistics jointly distributed U-statistics Lehmann 6.1 In this topic, we generalize the idea of U-statistics in two different directions. First, we consider single U-statistics for situations
More informationDIRECT AND INDIRECT LEAST SQUARES APPROXIMATING POLYNOMIALS FOR THE FIRST DERIVATIVE FUNCTION
Applied Probability Trust (27 October 2016) DIRECT AND INDIRECT LEAST SQUARES APPROXIMATING POLYNOMIALS FOR THE FIRST DERIVATIVE FUNCTION T. VAN HECKE, Ghent University Abstract Finite difference methods
More informationUniversity of California San Diego and Stanford University and
First International Workshop on Functional and Operatorial Statistics. Toulouse, June 19-21, 2008 K-sample Subsampling Dimitris N. olitis andjoseph.romano University of California San Diego and Stanford
More informationAn analogue of Rionero s functional for reaction-diffusion equations and an application thereof
Note di Matematica 7, n., 007, 95 105. An analogue of Rionero s functional for reaction-diffusion equations and an application thereof James N. Flavin Department of Mathematical Physics, National University
More informationInteractions. Yuan Gao. Spring Applied Mathematics University of Washington
Interactions Yuan Gao Applied Mathematics University of Washington yuangao@uw.edu Spring 2015 1 / 27 Nonlinear System Consider the following coupled ODEs: dx = f (x, y). dt dy = g(x, y). dt In general,
More informationNonlinear Systems and Control Lecture # 12 Converse Lyapunov Functions & Time Varying Systems. p. 1/1
Nonlinear Systems and Control Lecture # 12 Converse Lyapunov Functions & Time Varying Systems p. 1/1 p. 2/1 Converse Lyapunov Theorem Exponential Stability Let x = 0 be an exponentially stable equilibrium
More informationModeling and Experimentation: Compound Pendulum
Modeling and Experimentation: Compound Pendulum Prof. R.G. Longoria Department of Mechanical Engineering The University of Texas at Austin Fall 2014 Overview This lab focuses on developing a mathematical
More informationHomogeneous Equations with Constant Coefficients
Homogeneous Equations with Constant Coefficients MATH 365 Ordinary Differential Equations J. Robert Buchanan Department of Mathematics Spring 2018 General Second Order ODE Second order ODEs have the form
More informationApplied Mathematics Letters. Stationary distribution, ergodicity and extinction of a stochastic generalized logistic system
Applied Mathematics Letters 5 (1) 198 1985 Contents lists available at SciVerse ScienceDirect Applied Mathematics Letters journal homepage: www.elsevier.com/locate/aml Stationary distribution, ergodicity
More informationDYNAMICS IN 3-SPECIES PREDATOR-PREY MODELS WITH TIME DELAYS. Wei Feng
DISCRETE AND CONTINUOUS Website: www.aimsciences.org DYNAMICAL SYSTEMS SUPPLEMENT 7 pp. 36 37 DYNAMICS IN 3-SPECIES PREDATOR-PREY MODELS WITH TIME DELAYS Wei Feng Mathematics and Statistics Department
More informationLikelihood based Statistical Inference. Dottorato in Economia e Finanza Dipartimento di Scienze Economiche Univ. di Verona
Likelihood based Statistical Inference Dottorato in Economia e Finanza Dipartimento di Scienze Economiche Univ. di Verona L. Pace, A. Salvan, N. Sartori Udine, April 2008 Likelihood: observed quantities,
More informationInterval solutions for interval algebraic equations
Mathematics and Computers in Simulation 66 (2004) 207 217 Interval solutions for interval algebraic equations B.T. Polyak, S.A. Nazin Institute of Control Sciences, Russian Academy of Sciences, 65 Profsoyuznaya
More informationIterative Domain Decomposition Methods for Singularly Perturbed Nonlinear Convection-Diffusion Equations
Iterative Domain Decomposition Methods for Singularly Perturbed Nonlinear Convection-Diffusion Equations P.A. Farrell 1, P.W. Hemker 2, G.I. Shishkin 3 and L.P. Shishkina 3 1 Department of Computer Science,
More informationJean-Michel Billiot, Jean-François Coeurjolly and Rémy Drouilhet
Colloque International de Statistique Appliquée pour le Développement en Afrique International Conference on Applied Statistics for Development in Africa Sada 07 nn, 1?? 007) MAXIMUM PSEUDO-LIKELIHOOD
More informationNonlinear Dynamical Systems Lecture - 01
Nonlinear Dynamical Systems Lecture - 01 Alexandre Nolasco de Carvalho August 08, 2017 Presentation Course contents Aims and purpose of the course Bibliography Motivation To explain what is a dynamical
More informationTesting Algebraic Hypotheses
Testing Algebraic Hypotheses Mathias Drton Department of Statistics University of Chicago 1 / 18 Example: Factor analysis Multivariate normal model based on conditional independence given hidden variable:
More informationETNA Kent State University
Electronic Transactions on Numerical Analysis. Volume 4, pp. 373-38, 23. Copyright 23,. ISSN 68-963. ETNA A NOTE ON THE RELATION BETWEEN THE NEWTON HOMOTOPY METHOD AND THE DAMPED NEWTON METHOD XUPING ZHANG
More information8. Hypothesis Testing
FE661 - Statistical Methods for Financial Engineering 8. Hypothesis Testing Jitkomut Songsiri introduction Wald test likelihood-based tests significance test for linear regression 8-1 Introduction elements
More informationNontrivial Solutions for Boundary Value Problems of Nonlinear Differential Equation
Advances in Dynamical Systems and Applications ISSN 973-532, Volume 6, Number 2, pp. 24 254 (2 http://campus.mst.edu/adsa Nontrivial Solutions for Boundary Value Problems of Nonlinear Differential Equation
More information