APPROXIMATE SOLUTION OF A SYSTEM OF LINEAR EQUATIONS WITH RANDOM PERTURBATIONS

Transcription

1 APPROXIMATE SOLUTION OF A SYSTEM OF LINEAR EQUATIONS WITH RANDOM PERTURBATIONS P. Date paresh.date@brunel.ac.uk Center for Analysis of Risk and Optimisation Modelling Applications, Department of Mathematical Sciences, Brunel University, U.K. Abstract This work suggests a way of finding an approximate solution to a system of linear equations of the form AX = b, A = A + with a known square matrix A, a known vector b and a small structured random perturbation. Under certain realistic assumptions about the structure and the smallness of perturbations, it is shown that a uniformly convergent sequence of approximations to the random variable X = A b can be obtained in terms of polynomials of the random perturbations. Possible applications of this method in control theory are suggested. Numerical examples demonstrate the applicability of this method. 1 Introduction Many estimation and control problems rely on solution of linear least-squares problems or uncertain matrix inversion problems of the form AX = b. When the nominal data {A, b} are subject to uncertainty and/or disturbances, the performance of the optimal estimator may degrade appreciably. Similar problems arise while computing frequency response of a system with uncertain poles. Numerous approaches to alleviate this problem have been suggested. Min-max approach to least-squares problem under bounded data uncertainties has been studied in [1] and []. This approach formulates and solves problems of the form min X max AX b A A, b B where A and B are appropriate bounded uncertainty sets. Data-fitting under bounded uncertainties has corresponding author been considered in a more general setting in [3]. Applications of min-max based least-squares solutions have been reported in state estimation in general dynamical systems [4] and in water networks [5]. If statistical information is available about the uncertainty, the min-max based approach described above may be too conservative for use in certain applications. If the joint probability distribution function of all the uncertain parameters is known, the joint probability distribution of A b may be completely characterised using results in [6]. However, this level of information is rarely available. In this paper, the focus is to obtain computationally simpler estimates with a good average-case accuracy possibly at the expense of worst-case accuracy using substantially less information than the full probability density. To this end, this paper considers a sequence of uniform approximations to the true solution which is a random variable of the system of equations. The statistics of approximants is much simpler to compute than that of the true solution. The rest of the paper is organised as follows. The next section gives the exact assumptions and the problem formulation for the case when A is a square matrix and is invertible almost surely. Section 3 discusses the main results of this paper. Section 4 describes the case when the random perturbations have a special affine structure. Section 5 describes applications to analysis of uncertainty in frequency response of linear systems. Section 6 demonstrates these results with numerical examples. Problem formulation Notation in the paper is standard. Let R m n respectively, C m n denote the space of m n real

2 resp. complex matrices. Let R n denote the space of n vectors. I denotes identity matrix; its size is determined by the context. Ai, j denotes i, j th element of matrix A. Vectors will be denoted by boldface characters. In the problems considered in this paper, a system of linear equations of the following form is assumed to be given: AX = b, A = A +, A Q m m, b R m 1 where Q = R or C depending on context and is a real matrix-valued random variable satisfying P A < 1 = 1. Here denotes the maximum singular value and PΦ denotes the probability of occurence of event Φ. It is seen that the random perturbation is assumed to be small relative to the nominal value in a welldefined sense. Specific instances of the uncertainty set will be considered in section 4. 3 Main result Theorem 1 Suppose the given system of equations 1 satisfies and suppose that the inverse of A exists with probability 1. Define X = A b, X N = N A i A 3 with A = I. Then X N X with probability 1. Proof : The proof rests on the following standard result in probability see, e.g., [7], theorem 7.4: If N P X N X > ɛ < for all ɛ >, then X N X with probability 1. The proof is based on deriving an upper bound on P X N X > ɛ and then showing that summation of these upper bounds over N is finite. 1 Since P A < 1 = 1, the following power series expansion holds with probability 1 see, e.g. [8], chapter 5: A + = I + A = A A i A 1 From proof of [7], theorem 7.4, it is easy to see that analogous result holds for vector-valued random variables. so that P X = A i A = 1. Suppose that P A α = 1 for some α < 1. Also, let A = β. For a given ɛ >, let N ɛ be the smallest integer such that αnɛ+1 β 1 α ɛ. Now, using definition of X N, XNɛ > ɛ { = P i=n ɛ A i } A > ɛ =, since, with probability 1, 4 { } A i A i=n ɛ Next, for any ɛ >, N= N ɛ XN > ɛ N= since N ɛ completes the proof. αnɛ+1 β 1 α ɛ. 5 XN > ɛ <, 6 log ɛ1 α αβ logα is finite for any ɛ >. This The above result shows a way of obtaining a series of random vectors which converges with probability 1 to the true solution of the system of equations. The expected value of X N is given by N EX N = E A i A. This is a function of the first N moments of random variable and may thus be found using a limited information about the uncertainty. In the next section, with a special structure is considered, which yields particularly simple low order approximations. 4 Affinely parameterised perturbations Consider the system 1 with which satisfies and has the following structure: = a i A i, A i R m m, 7

3 and a i are scalar random variables with Ea i =, i = 1,,..., k. The matrices A i are in general, sparse matrices, with 1 s in places where the random parameter a i has an impact on the nominal entry in the matrix A and zeros at all other elements numerical examples will make this point clear. The assumption that a i are zero-mean random variables is perfectly reasonable and implies that the mean is absorbed in A. It is possible to obtain useful low order approximations may be obtained with a very small number of terms for this structure, as explained below. 1. The first order approximation X 1 = I a i A A i A is a linear combination of zero mean random variables a i and is of independent interest. If the variance of a i are known and if N is sufficiently large, this expression may be used alongwith central limit theorem to build approximate confidence intervals around each element of the nominal th order solution A. If N is small but the upper and the lower bounds on a i are known, Hoeffding s inequality [9] may be used to obtain potentially conservative approximate error bounds. See [1] for a recent application of Hoeffding s inequalities in engineering. Alternatively, X 1 may simply be used to test the sensitivity of the solution of a system of linear equations to small perturbations in certain entries in the matrix without having to solve the perturbed system.. If a i are uncorrelated, it is easy to show that EX = I + Ea i A A i A. Note that the computation for EX does not require sampling the distribution of a i. For a large number of uncertain parameters, the cost-saving to build an accurate estimate of the average value of A b may be significant. Simulation experiments indicate that higher order approximations indeed tend to be more accurate than the nominal th order solution A. This is demonstrated in section 6 through an example. If first 4 moments of a i are known, it is possible to obtain covariance matrix of X. The expression for covariance is straighforward if tedious and is ommitted. It is necessary here to comment on the relationship between the order of approximation and the size of perturbations. The bound on the relative size of perturbation A will not be known in general. However, this bound does not appear in X N itself. If X N for some N 1 differs substantially by more than 1%, say from X = A, it may indicate that the small perturbations condition may be violated and the data is not reliable enough to build an estimate. In general, it may be seen from the proof of theorem 1 that X N X X A N+1 1 α holds with probability 1. The choice of N then depends on the trade-off between tractability and a priori knowledge of the size of uncertainty. Finally, the results presented here may be trivially extended to the case when b = b + δ b for a random perturbation δ b which is uncorrelated to. 5 Applications The technique presented in this paper is generic and may have a wide variety of applications. Two applications are described in some detail here. The focus of the first application is to map the available information about uncertainty in parameters of a linear system into corresponding uncertainty in its frequency response. The second application relates to perturbation of a transfer function of a controller and its effect on the closed-loop transfer functions. 5.1 Uncertainty in frequency response of a linear system due to parametric perturbations If the parametric uncertainty in a model of a linear system is in the form of a covariance matrix of parameter estimates, there are numerous other ways of mapping it into frequency response; see [11] and references therein. However, the parameters may be obtained by physical knowledge of the underlying dynamics, e.g. by knowing the values of nomial motor resistance and inductance in a DC drive. The motivation of this application is the latter situation, when the parameters may be known within certain tolerances around their nominal values.

4 Consider a continuous time linear, shift-invariant system in a standard notation, dx = Axt + But, dt y k = Cxt + Dut where A = A +, A is a constant real matrix and is a matrix-valued real random variable satisfying P jωi A < 1 = 1, ω. 8 The perturbation may represent the uncertainty in physical parameters which yield the poles of the system. Following the steps of theorem 1, the first order approximation of the uncertain frequency response P jω = CjωI A B + D at a frequency ω is given by P 1 = CjωI A B + D CjωI A jωi A B. 9 Unlike P, P 1 is affine in uncertainty. If the distribution of this uncertainty is known, approximate confidence intervals of pointwise frequency response of the system may be easily obtained from Uncertainty in closed-loop frequency response due to controller perturbations Consider a closed-loop described by the following equations: y = P u + w u = Cy. 1 Here, P is the plant, C is the controller, w is a disturbance acting on the output and u and y are the plant input and plant output respectively. The plant and the controller are assumed to be matrix valued linear operators while the signals are assumed to be vector-valued. It may be easily shown that the transfer function from w to [ y T u ] T T is given by [ [ ] y I P C = u] CI P C w. 11 Suppose the controller is given by C = C + where C is a known transfer function and is a zero mean random perturbation e.g. accounting for tolerances of nominal values of electrical or mechanical components used in hardware implementation. It is of interest to find the effect of this perturbation on the frequency response of the transfer function matrix. For notational simplicity, let S = I P C and let I P C I P C T = CI P C, T = C I P C. It is assumed that P S P jω < 1 = 1 holds at each frequency ω of interest. Using the first order approximation for I P C and then retaining only the first order terms in in the resulting expression yields after some elementary manipulation [ ] S T T + P S I C P. 1 S This expression may be used to find pointwise confidence intervals for the frequency response of the transfer matrix to perturbations in the controller transfer function. 6 Numerical Examples Results of some numerical experiments related to better average case accuracy of higher order approximations are presented here. Specific applications, such as the ones described in the last section, are not discussed due to space constraints. Consider a simple linear system of equations A + X = b with, [ ] [ 1 1 A =, b =, = a 3 3 5] 1 A 1 + a A, 1 A 1 =, A = 1 and a i P. a 1 is assumed to be uniformly distributed in [.4, +.4] and a is assumed to be uniformly distributed in [.6, +.6]. This gives a variation of ±% around nominal values of the corresponding two entries in A. 1 samples of each a 1, a are generated and expected value of solution E X = [ ] T is computed. The nominal solution in this case is A = [ ] T. Using the result described earlier, the expected value of second order approximation is computed as EX = A + Ea 1 A A 1 + Ea A A A = [ ] T.

5 Note that first order moments are zero and the corresponding term need not be computed. It is seen that using a simple second order approximation with a modest extra computation yields a clear improvement in the accuracy of solution in this case. As another example, consider an iterative numerical experiment. An i th iteration proceeds as follows. A random matrix A of size and a random vector of size 1 is generated. Two random variables are considered: a 1 is normally distributed with Ea 1 = and Ea 1 =. A 1, and a is uniformly distributed with Ea = and Ea =.1 A,. For 1 realisations of a 1 and a, 1 A = A + a 1 + a 1 is computed and the mean solution E X = EA b is computed. Care taken was to ensure that the matrix A does not become too illconditioned. The solutions EX and X = A are computed. The following quantities are taken as measure of errors: L 1 = EX E X, and L = X E X. This entire experiment is repeated 1 times. It was found that L 1 < L held for 94 out of 1 times. In the remaining instances the assumption A < 1 was violated more than once. It is thought that this presents a convincing case for using a higher order approximation when better average case accuracy is required. 7 Conclusion A method to build a sequence of random vectors which converges almost surely to the solution of a system of linear equations under random perturbation is suggested. Probabilistic description of N th vector in this sequence may serve as an approximate description of the actual solution. When information about higher order moments for random perturbations is available, it may be used to build an accurate approximation to the expected value of the solution as well as to the confidence intervals around this expected value. The extension of these results to least squares problems with non-square matrices and examples of suggested applications will be reported elsewhere. References [1] S. Chandrasekaran, G. Golub, M. Gu, and A. Sayed, Parameter estimation in the presence of bounded data uncertaities, SIAM J. Matrix Anal. and Appl., vol. 19, pp. 35 5, [] L. E. Ghaoui and H. Lebret, Robust solutions to least-squares problems with uncertain data, SIAM J. Matrix Anal. Appl., vol. 18, pp , [3] G. Watson, Data fitting problem with bounded uncertainties in the data, SIAM J. Matrix Anal. Appl., vol., pp , 1. [4] A. Sayed, A framework for state space estimation with uncertain models, IEEE Trans. Automat. Contr., vol. 46, pp , 1. [5] A. Nagar and R. Powell, LFT/SDP approach to the uncertainty analysis for state estimation of water distribution systems, IEE Proceedings, vol. 149, pp ,. [6] J.Feinberg, On the universality of the probability distribution of the product B X of random matrices. [7] G. Grimmett and D. Stirzaker, Probability and Random Processes. Oxford University Press, 1. [8] R. Horn and C. Johnson, Matrix Analysis. Cambridge University Press, [9] W. Hoeffding, Probability inequalities for sums of bounded random variables, Amer. Statistical Asso. Journal, no. 3, pp. 13 3, [1] F. Paganini, A set-based approach for white noise modeling, IEEE Trans. Automat. Contr., vol. 41, pp , [11] X. Bombois, M. Gevers, G. Scorletti, and B. Anderson, Robustness analysis tools for an uncertainty set obtained by prediction error identification, Automatica, vol. 37, pp , 1.