SAZ3C NUMERICAL AND STATISTICAL METHODS Unit : I -V

UNIT-I Introduction Mathematical Preliminaries Errors: Computations, Formula Errors in a Series Approximation Roots of Equations Linear Equations Bisection method False Position Method Newton-Raphson Method Secant Method Muller s Method Lin-Bairstow s Method Simultaneous Linear Equations Matrix Inversion Method Gauss Elimination Gauss-Jordan LU Decomposition Methods Gauss-Seidel Method 2

Introduction The problems of obtaining solutions of equations of the form f(x)=0. In other words we have to find a number x0 such that f(x0)=0. If f(x) is a polynomial then the equation f(x) is called an algebraic equation Transcendental equations: Equations which involves transcendental functions like sin x, cos x, tan x, log x, exp x etc. are called transcendental equations. 3

Errors ABSOLUTE ERROR If α is an approximate value of a quantity whose exact value is a, then the difference is called the absolute error of α or simply the error of α EXPERIMENTAL ERROR An experimental error is an error present in the given data. Such errors may arise from measurements. TRUNCATION ERROR In an iterative computational method the sequence of computational steps necessary to produce an exact result is truncated after a finite number of steps. An error arising out of such truncation is called truncation error. ROUNDING OFF ERRORS The process of cutting of digits and retaining required number of digits is called rounding off. Errors arising from the process of rounding off during computation are called round-off errors. a 4

Bisection Method Let f(x) be a continuous function defined on [a,b] such that f(a) and f(b) are of opposite signs. Hence one root of the equation f(x)=0 lies between a and b. For definiteness we assume that f(a)<0 and f(b)>0. STEP 1: a b be the first approximation of the required root ( x1 is the 2 midpoint of a and b) Let x1 STEP 2: If f(x1) then x1 is a root of f(x). If not the lies between a and x1 or x1 and b depending on whether f(x1)>0 or f(x1)<0. STEP 3: Bisect the interval in which the root lies and continue the process until the root is found to the desired accuracy. 5

Muller Method The method consists of deriving the coefficients of parabola that goes through the three points: 1. Write the equation in a convenient form: f 2 ( x) a( x x2 ) b( x x2 ) c 2 2. The parabola should intersect the three points [xo, f(xo)], [x1, f(x1)], [x2, f(x2)]. The coefficients of the polynomial can be estimated by substituting three points to give f ( xo ) a( xo x2 ) 2 b( xo x2 ) c f ( x1 ) a( x1 x2 ) 2 b( x1 x2 ) c f ( x2 ) a ( x2 x2 ) 2 b ( x2 x2 ) c 6

Muller method 3.Three equations can be solved for three unknowns, a, b, c. Since two of the terms in the 3rd equation are zero, it can be immediately solved for c=f(x2). f ( xo ) f ( x2 ) a( xo x2 ) 2 b( xo x2 ) f ( x1 ) f ( x2 ) a( x1 x2 ) 2 b( x1 x2 ) If h o x1 - x o o h1 x 2 - x1 f ( x1 ) f ( xo ) x1 xo 1 f ( x2 ) f ( x1 ) x2 x1 ( ho h1 )b ( ho h1 ) 2 a ho o h1 1 h1b h12 a h1 1 a 1 o h1 ho b ah1 1 c f ( x2 ) 7

Muller Method Cont Roots can be found by applying an alternative form of quadratic formula: 2c x3 x2 b b 2 4ac The error can be calculated as x3 x2 a 100% x3 ±term yields two roots, the sign is chosen to agree with b. This will result in a largest denominator, and will give root estimate that is closest to x2. Once x3 is determined, the process is repeated using the following guidelines: 1. If only real roots are being located, choose the two original points that are nearest the new root estimate, x3. 2. If both real and complex roots are estimated, employ a sequential approach just like in secant method, x1, x2, and x3 to replace xo, x1, and x2. 8

Bairstow Method Bairstow s method is an iterative approach loosely related to both Müller and Newton Raphson methods. It is based on dividing a polynomial by a factor x-t: f n ( x) ao a1 x a2 x 2 an x n f n 1 ( x) b1 b2 x b3 x 2 bn x n with a reminder R b o, the coefficien ts are calculated by recurrence relationship bn an bi a i bi 1t i n 1 to 2 9

Bairstow Method To permit the evaluation of complex roots, Bairstow s method divides the polynomial by a quadratic factor x2-rx-s: f n 2 ( x) b2 b3 x bn 1 x n 3 bn x n 2 R b1 ( x r ) bo Using a simp le recurrence relationship bn an bn-1 an-1 rbn bi ai rbi 1 sbi 2 i n- 2 to 0 For the remainder to be zero, bo and b1 must be zero. However, it is unlikely that our initial guesses at the values of r and s will lead to this result, a systematic approach can be used to modify our guesses so that bo and b1 approach to zero. 10

Newton Raphson method Step 1. Write out f(x). Step 2. Find f'(x), the derivative of f(x). f(x n ) Step 3. Find x n+1 =x n. f'(x n ) Step 4. Repeat Step 3 for more accuracy using the x n+1 found in Step 3 as x n. The more times Step 4 is repeated the more accurate the answer. 11

Secant Method The Newton s Method requires 2 function evaluations (f,f ). The Secant Method requires only 1 function evaluation and converges as fast as Newton s Method at a simple root. Start with two points x0,x1 near the root (no need for bracketing the root as in Bisection Method or Regula Falsi Method) xk-1 is dropped once xk+1 is obtained. Gauss Elimination Method Using row operations, the matrix is reduced to an upper triangular matrix and the solutions obtained by back substitution 12

LU-Decomposition 13

Matrix Inverse Using LUDecomposition LU decomposition can be used to obtain the inverse of the original coefficient matrix. Each column j of the inverse is determined by using a unit vector (with 1 in the jth row ) as the RHS vector Order of Convergence # Bisection method p = 1 ( linear convergence ) # False position - generally Super linear ( 1 < p < 2 ) # Secant method # Newton Raphson method p = 2 quadratic 1 5 1.618 2 (super linear) 14

Gauss Jordan Method 15

Gauss Seidal Method 16

UNIT-II Numerical Differentiation Errors in Numerical Differentiation Cubic Spline Method Numerical Integration Trapezoidal Rule Simpson s 1/3 and 3/8 Rules Romberg Integration Ordinary Differential Equations Taylor s Series Method Euler s Method Runge-Kutta 2nd and 4th Order Methods Predictor-Corrector Methods. 17

Errors in Numerical Differentiation 19

Cubic spline Method 20

Numerical Integration The process of evaluating a definite integral from a set of tabulated values of the integrand f(x), which is not known explicitly is called numerical integration. Trapezoidal Rule Simpson s 1/3 and 3/8 Rules Romberg Integration 21

Trapezoidal Rule 22

Simpson s 3/8 rule xn x0 3h y0 yn 3 y1 y2 y4 y5 yn 1 f x dx 2 y y y 8 3 6 9 where h is the width of the int erval. 24

Romberg s Method Romberg s method can be used to find a better approximation to the results obtained by the finite differences method. ROMBERG S FORMULA: I 2 I1 I I2 3 25

Taylor series Method Consider the first order differenti al equation dy f x, y dx Then, the formula for Taylor ' s series method is given by h ' h 2 '' h3 ''' y x1 h y x2 y2 y1 y1 y1 y1 1! 2! 3! 26

Euler Method Euler ' s formula is given by yn 1 yn hf xn, yn, n 0,1,2,3 Runge Kutta Method First Order R-K Method: The sec ond order Runge Kutta formula 1 k1 k 2 2 where, k1 h f x0, y0 is y1 y0 k 2 h f x0 h, y0 k1. 27

Second order Runge Kutta Method 28

Fourth order Runge Kutta Method 29

Predictor-Corrector Formula MILNE S PREDICTOR FORMULA IS y n 1, p y n 3 4h ' ' ' 2 yn 2 yn 1 2 yn 3 MILNE S CORRECTOR FORMULA IS y n 1,c y n 1 h ' y n 1 4 y n' y n' 1 3 30

UNIT-III Sampling Frequency Distribution Cumulative Frequency Function Grouped Sample Measures of Central Tendency Mean Median and Mode Geometric Mean Harmonic Mean Dispersion Range Mean Deviation Variance Standard Deviation Moments- Computation of Moments. 31

SAMPLING Sampling is the process of selecting a small number of elements from a larger defined target group of elements such that the information gathered from the small group will allow judgments to be made about the larger groups Basic Terminologies : Population: The entire group under study is called population. Sometimes it is also called as the universe. Sample: A subset of the population that should represent the entire group is called Sample. 32

FREQUENCY DISTRIBUTION Frequency Distribution is simply a table in which the data are grouped into classes and the number of cases which fall in each class is recorded. TYPES : Discrete frequency distribution. Continuous frequency distribution. Cumulative frequency distribution. Bivariate frequency distribution. 34

MEASURES OF CENTRAL TENDENCY MEANING A CENTRAL VALUE IS A SINGLE VALUE WHICH DESCRIBES THE CHARCTERISTICS OF THE ENTIRE DATA. MEASURES ARITHMETIC MEAN MEDIAN MODE HARMONIC MEAN GEOMETRIC MEAN 35

ARITHMETIC MEAN Arithmetic mean is usually called as average and is given by sum of all observations divided by the total number of observations given. 36

MEAN ARITHMETIC MEAN GEOMETRIC MEAN x HARMONIC MEAN 37

MEDIAN The median is a simple measure of central tendency. To find the median, we arrange the observations in order from smallest to largest value. If there is an odd number of observations, the median is the middle value. If there is an even number of observations, the median is the average of the two middle values. 38

MODE For various methods of calculating mean, median, mode for different types of data, follow the link https://sol.du.ac.in/mod/book/view.php?id=1317&chapterid=1065 39

MEASURES OF DISPERSION The average measures the center of the data. Another feature of the observation is how the observations are spread about the center. The observations may be close to the center or they may be spread away from the center. If the observations are close to the center (usually the arithmetic mean or median), we say that dispersion, scatter or variation is small. If the observations are spread away from the center, we say dispersion is large. TYPES RANGE QUARTILE DEVIATION MEAN DEVIATION STANDARD DEVIATION 40

MEASURES OF DISPERSION ABSOLUTE MEASURE OF DISPERSION : It measures the distribution in original units of data. Variability in two or more series can be compared, provides they are in the same unit and same average. RELATIVE MEASURE OF DISPERSION : It is free from unit of measurement of data. It is the ratio of the measure of absolute dispersion to the average, from which absolute deviations are made. It is called as coefficient of variation. For formulae, merits and demerits of various measures of dispersion refer http://www.yourarticlelibrary.com/statistics-2/4-commonly-used-measures-ofdispersion-statistics/92258 41

MOMENTS Definition: Moments can be defined as the arithmetic mean of various powers of deviation taken from the mean of a distribution. Moment is denoted by greek letter µ. The mean of the first power is called first moment µ1: mean of the second power is called second moment µ2 and so on. 42

MOMENTS 43

MOMENTS 44

Coefficient Of Skewness And Kurtosis Using Moments 45

UNIT-IV Probability Characteristics: Addition, Multiplication Conditional Probability Laws Discrete Distributions Random Variable Density and Distribution Functions Binomial Distribution Poisson Distribution Hypergeometric Distribution Mathematical Expectation. 46

Probability The numerical chance that a specific outcome will occur. i.e.)., it is a mathematical measure of Measuring the certainty or uncertainty of an event Sometimes you can measure a probability with a number: "10% chance of rain", or you can use words such as impossible, unlikely, possible, even chance, likely and certain. Example: "It is unlikely to rain tomorrow". 47

Basic Definitions Experiment: act or process that leads to a single outcome that cannot be predicted with certainty Sample space: all possible outcomes of an experiment. Denoted by S Event: any subset of the sample space S typically denoted A, B, C, etc. Simple event: event with only 1 outcome Null event: the empty set F Certain event: S Laws of Probability 1. 0 P(A) 1 for any event A 2. P( ) = 0, P(S) = 1 and P(A ) = 1 P(A) 4. If A and B are disjoint events, then P(A B) = P(A) + P(B) 5. If A and B are independent events, then P(A B) = P(A) P(B) 6. For any two events A and B,P(A B) = P(A) + P(B) P(A B) 48

Additive Rule & Multiplicative Rule For any two events, A and B, the probability of their union, P( A B) is P( A B) P( A) P( B) P( A B) When two events A and B are mutually exclusive, P(A B) = 0 and P(A B) = P(A) + P(B). 49

Independent Event &Conditional Probabilities 1.Two events, A and B, are said to be independent if the occurrence or nonoccurrence of one of the events does not change the probability of the occurrence of the other event. interms of conditional, Two events A and B are independent if and only if P(A B) = P(A) or P(B A) = P(B) Otherwise, they are dependent. 2. The probability that A occurs, given that event B has occurred is called the conditional probability of A given B and is defined as P( A B) P( A B) if P( B) 0 P( B) 50

Random Variable RANDOM VARIABLE Let S be a sample space corresponding to the random experiment E. A function X defined on the sample space S to the real number system R is called a random variable. DISCRETE RANDOM VARIABLES If X is an random variable which can take a finite number of countable infinite number of values, X is called a discrete random variables. CONTINUOUS RANDOM VARIABLES If X is an random variable which can takes all values (i.e., infinite number of values) in an interval, then X is called a Continuous random variables. 51

Examples of Random Variables DISCRETE RANDOM VARIABLES Number of sales Number of calls Shares of stock People in line Mistakes per page CONTINUOUS RANDOM VARIABLES Length Depth Volume Time Weight 52

Probability Distribution A probability distribution (function) is a list of the probabilities of the values (simple outcomes) of a random variable.if the outcomes are of discrete in nature, then the corresponding probability distribution is called as probability mass function(pmf) Table: Number of heads in two tosses of a coin y P(y) outcome probability 0 1/4 1 2/4 2 1/4 For some experiments, the probability of a simple outcome can be easily calculated using a specific probability function. If y is a simple outcome and p(y) is its probability.the following are the properties of a probability function, 0 p( y ) 1 p( y) 1 all y 53

Probability Density Function A function which integrates 0 to 1 over its range and from which event probabilities can be determined.it is associated with continuus random variable. f(x) Area under curve sums to one. Random variable range A theoretical shape - if we were able to sample the whole (infinite) population of possible values, this is what the associated histogram would look like. 54

Cumulative Distribution Function Cumulative Distribution Function (CDF) of a random variable X is defined by F ( y ) P(Y y ) P(X<x) Pr operties (i ) F (b) P(Y b) b p( y ) y (ii ) F ( ) 0 & F ( ) 1 (iii ) F ( y ) is monotonically increasing in y 55

Binomial Distribution The experiment consists of n identical trials (simple experiments). Each trial results in one of two outcomes (success or failure) The Binomial Probability Distribution is n x n x P( x) p q x 56

Moments of Binomial Distribution Mean Variance np 2 npq Standard Deviation npq As the n goes up, the distribution looks more symmetric and bell shaped. 57

Poisson Distribution A random variable is said to have a Poisson Distribution with rate parameter, if its probability function is given by: P( y ) y y! e, for y 0,1,2,... The Moments are (I ) Mean = = variance (ii) MGF = e ( et 1) 58

The Hyper geometric Distribution The hypergeometric random variable is the number of successes, x, drawn from the r available in the n selections. r N r x n x P ( x ) N n where N = the total number of elements r = number of successes in the N elements n = number of elements drawn X = the number of successes in the n elements 59

Expected Value and Variance All probability distributions are characterized by an expected value and a variance (standard deviation squared). Discrete case: E( X ) x all i p(xi ) i p(xi )dx x Continuous case: E( X ) x all x 60

Properties of Expectation If c= a constant number (i.e., not a variable) and X and Y are any random variables 1. E(c) = c 2. E(cX)=cE(X) 3. E(c + X)=c + E(X) 4. E(X+Y)= E(X) + E(Y) 61

UNIT-V CORRELATION AND REGRESSION ANALYSIS. COEFFICIENT OF CORRELATION LINEAR LEAST SQUARES FIT. NONLINEAR FIT FITTING A POLYNOMIAL FUNCTION PROPERTIES MULTIPLE CORRELATION PARTIAL CORRELATION RANK CORRELATION TESTS OF SIGNIFICANCE CHI SQUARE TEST GOODNESS OF FIT ALGORITHM AND ANALYSIS OF CONTINGENCY TABLES T-TEST AND F-TEST. 62

CORRELATION Correlation is a statistical method used to study the relationship between two or more variables. 63

TYPES OF CORRELATION 64

Correlation Coefficient Correlation coefficient measures the direction and degree of relationship between the variables in one figure. Properties : It is independent of the choice of both origin and the scale of etween observation. It is independent of the unity of measurement. It lies between -1 and +1. 65

Methods Of Studying Correlation Scatter diagram method. Scatter diagram is a graphical method of finding correlation. It is one of the simplest procedures to judge correlation between variables. One variable is taken along the x- axis and other variable is taken along the y-axis. From the plotted points, we can find whether the variables are correlated or not. 66

KARL PEARSONS METHOD Karl Pearson s Coefficient of Correlation is widely used mathematical method wherein the numerical expression is used to calculate the degree and direction of the relationship between linear related variables. Pearson s method, popularly known as a Pearsonian Coefficient of Correlation, is the most extensively used quantitative methods in practice. The coefficient of correlation is denoted by r. For example refer, http://www.stat.wmich.edu/s216/book/node122.html 67

RANK CORRELATION This method is based on Ranks. It is useful when the data is of qualitative nature like honesty, efficiency, intelligence, beauty etc., The rank correlation coefficient is given by D 2 1 N ( N 2 1) Where N is the number of observations and D is the difference between number of observations 68

REGRESSION Regression is a statistical method which is used to estimate the unknown value of one variable using known value of other variable, provided the variables are correlated. 69

REGRESSION EQUATIONS Regression Line of X on Y: Regression Line of Y on X : 70

Tests of hypothesis Statistical hypotheses are statements about probability distributions of the populations. In simple, the assumptions or guesses about the population involved, which may or may not be true are called statistical hypotheses. Decisions about the populations on the basis of sample information are called statistical decisions. 71

Tests of hypothesis Goal: Make statement(s) regarding unknown population parameter values based on sample data Elements of a hypothesis test: Null hypothesis - Statement regarding the value(s) of unknown parameter(s). Typically will imply no association between explanatory and response variables in our applications (will always contain an equality) Alternative hypothesis - Statement contradictory to the null hypothesis (will always contain an inequality) Test statistic - Quantity based on sample data and null hypothesis used to test between null and alternative hypotheses Rejection region - Values of the test statistic for which we reject the null in favor of the alternative hypothesis 72

Errors 73

t - test When the sample size is 30 or when the population standard deviation is not known, the student s t distribution is used. The t- statistic is defined as t = X-μ S/ n Where S = Σ(X-X)2 n-1 for (n-1) degrees of freedom. PROPERTIES The t-distribution is symmetrical and has a mean zero like the standard normal distribution. The variable t- diatribution ranges from minus infinity to plus infinity The variance of the t- distribution is greater than one, but approaches one, as the degree of freedom and hence the sample size becomes large. For example, refer https://web.mst.edu/~psyworld/texample.htm 74

F- test (Variance - ratio test) The objective of F-test is to find Whether the two samples drawn from a normal population have the same variance. Whether two independent estimates of population variance differ significantly. The F- statistic is defined as 2 2 F= S1 /S2 2 2 2 2 Where S1 = Σ(x1-x1) /n1-1 ; S2 = Σ (x2-x2) /n2-1 Where n1and n2 refers to the number of items in the sample I and II. It should be noted that the numerator is always greater than the denominator. For example, refer http://formulas.tutorvista.com/math/f-test-formula.html 75

Chi Square Test The χ2 test (Pronounced as chi square) is one of the most widely used non-parametric tests in statistical work. It is defined as 2 ( ) O E 2 E where O is the observed frequency and E is the expected frequency. For problems, refer http://stattrek.com/chi-square-test/independence.aspx?tutorial=ap http://www.statisticshowto.com/probability-and-statistics/chi-square/ 76