Hani Mehrpouyan, California State University, Bakersfield. Signals and Systems

Transcription

1 Hani Mehrpouyan, Department of Electrical and Computer Engineering, California State University, Bakersfield Lecture 1 (Intro, History and Background) April 4 th, 2013 The material in these lectures is partly taken from the books: Elementary Numerical Analysis, 3rd Edition, K. Atkinson, W. Han, Applied Numerical Methods Using MATLAB, 1st Edition, W. Y. Yang, W. Cao, T.-S. Chung, J. Morris, and Applied Numerical Analysis Using MATLAB, L V. Fausett. 1

2 Outline What to expect from the course Evaluation Homework Labs Midterms Final exam History and Introduction 2

3 Instructor Raised in British Columbia, Canada. Completed Ph.D. in August of 2010, at Queen s University. Completed Post-Docs at Chalmers University of Technology and the University of Luxembourg. Currently an assistant professor at CSUB. Research Interests: Application of signal processing algorithms in communication systems. Cooperative communication systems. Heterogeneous networks. Synchronization and channel estimation. Millimeter wave communication systems. Large-scale multiple antenna wireless systems. 3

4 Lectures Thursdays 12:00-14:00 PM 14:30-16:30 PM Labs Monday 14:00 PM-16:00 PM 4

5 Mission Students in this course will gain a comprehensive knowledge of number representation and basic concepts of error; numerical solutions of nonlinear equations and systems of equations; interpolation and extrapolation; numerical differentiation and integration. 5

6 Text Books Required Book: Elementary Numerical Analysis, 3rd Edition, Kendall Atkinson, Weimin Han, ISBN: Lots of solved problems which is key to grasping the topics. Highly recommended: Schaum's Outline of Numerical Analysis Francis Scheid, ISBN-10: ISBN-13: Recommended: Applied Numerical Methods Using MATLAB, 1st Edition, Won Y. Yang, Wenwu Cao, Tae-Sang Chung, John Morris, ISBN: Lots of Matlab solutions Recommended: Applied Numerical Analysis Using MATLAB Laurene v. Fausett, ISBN:

7 Website Means of Communication Lectures, labs, all materials related to the course will be posted here. I will try to post the slides before the lectures. Office hours They are scheduled per request. You can me at any time and I will try to do my best to answer your question or schedule you in for a meeting. Hani.mehr@ieee.org 7

8 Phone Means of Communication You can call my office phone at any time. If I am available we can discuss any problem or issue you may be facing

9 15% of your total mark. Homework Homework is due on every Thursday at the beginning of each lecture. Homework 1 will be posted and is due on the 18 th of April. The solutions will be posted on the course website. Based on general problems and may not be from the textbook. Late homework will not be accepted no exceptions applied. I am sorry but we need to be fair. The following constitutes as a late submission : Hand-in your homework at the end of the lecture. Hand-in your homework at the end of the day. Hand-in your homework because you are sick with no Dr. s note. 9

10 20% of your total mark. There are a total of 5 labs. Labs These will be mainly Matlab projects on the different topics in the course. Labs will be posted 2-3 days before the actual date. The lab report which includes your outcomes and explanations is due at the beginning of the next lab. Your mark is based on the thoroughness of your report and analysis, grammatical correctness, and the report s overall professionalism. Again no late reports are accepted. The same conditions as before apply here. 10

11 Time line 11

12 Time line 12

13 Time line 13

14 Time line 14

15 Numerical Analysis Numerical analysis is the area of mathematics and computer science that creates, analyzes, and implements algorithms for solving numerically the problems of continuous mathematics. During the past half-century, the growth in power and availability of digital computers has led to an increasing use of realistic mathematical models in science and engineering. 15

16 Numerical Analysis The formal academic area of numerical analysis varies from quite theoretical mathematical studies to computer science issues. 16

17 History Numerical algorithms are almost as old as human civilization. The Rhind Papyrus (1650 BC) of ancient Egypt describes rootfinding. Archimedes of Syracuse ( BC) created method of exhaustion for calculating lengths, areas, and volumes of geometric figures. This approach was very similar to numerical integration and it was an important precursor to the development of the calculus by Isaac Newton and Gottfried Leibnitz. 17

18 History A major impetus to developing numerical procedures was the invention of calculus by Newton and Leibnitz, as this led to accurate mathematical models for physical reality. These mathematical models cannot usually be solved explicitly, and numerical methods to obtain approximate solutions are needed. Another important aspect of the development of numerical methods was the creation of logarithms by Napier (1614) and others, giving a much simpler manner of carrying out the arithmetic operations of multiplication, division, and exponentiation. 18

19 History Newton created a number of numerical methods for solving a variety of problems, and his name is attached today to generalizations of his original ideas. Rootfinding Polynomial interpolation The giants of mathematics of the 18th and 19th centuries made major contributions to the numerical solution of mathematical problems. Foremost among these are Leonhard Euler ( ), Joseph-Louis Lagrange ( ), and Karl Friedrich Gauss ( ). 19

20 Overview One of the most important and influential of the early mathematical models in science was that given by Newton to describe the effect of gravity: where m e is the mass of the Earth, r is the distance between the centers of the two bodies, and G is the universal gravitational constant. 20

21 Overview Newton s model of gravitation has led to many problems that require solution by approximate means, usually via numerical solution of ordinary differential equations. Newton s model was applied to give mathematical models for solid and fluid mechanics. Civil and Mechanical Engineering use these models as the basis for most modern work on solid structures and the motion of fluids. 21

22 Overview Numerical analysis has become a basic part of the work of researchers in these areas of engineering. 22

23 Numerical linear and nonlinear algebra. This refers to problems involving the solution of systems of linear and nonlinear equations, possibly with a very large number of variables. Many problems in applied mathematics involve solving systems of linear equations, with the linear system occurring naturally in some cases and as a part of the solution process in other cases. 23

24 Num. lin. & nonlin. algebra. Linear systems are usually written using matrix-vector notation, Ax=b, with A the matrix of coefficients for the system, x the column vector of the unknown variables x 1,..., x n, and b a given column vector. Solving linear systems with up to a n = 1000 variables is now considered relatively easy. Via Gaussian elimination and its variants. 24

25 Num. lin. & nonlin. algebra. For larger linear systems, there are a variety of approaches depending on the structure of A. Direct methods lead to a theoretically exact solution x in a finite number of steps, e.g., Gaussian elimination. Result in rounding errors in practice. Iterative methods are approximate methods which create a sequence of approximating solutions of increasing accuracy. Nonlinear problems are often treated numerically by reducing them to a sequence of linear problems. Rootfinding. 25

26 Approximation Theory This category covers the approximation of functions and methods based on using such approximations. When evaluating a function f(x) with x a real or complex number, keep in mind that a computer or calculator can only do a finite number of operations, e.g., addition, subtraction, multiplication, and division, together with comparison operations. With the four basic arithmetic operations, we can evaluate polynomials 26

27 Approximation Theory The evaluation of all other functions, e.g. f(x) = x 1/2 or 2 x, must be reduced to the evaluation of a polynomial or rational function that approximates the given function with sufficient accuracy. All function evaluations on calculators and computers are accomplished in this manner. One method of approximation is called interpolation. 27

28 Solving diff. and integral eq. Most mathematical models used in the natural sciences and engineering are based on ordinary differential equations. The numerical methods for these equations are primarily of two types. The first type approximates the unknown function in the equation by a simpler function, e.g., polynomial. Finite element method. The second approach approximates the derivatives or integrals by the solution functions at a discrete set of points. Most initial value problems for ordinary differential equations and partial differential equations are solved in this way. 28

29 Effects of computer hardware Virtually all numerical computations are carried out on digital computers. Their structure and properties affect the structure of numerical algorithms, especially when solving large linear systems. Historically, computer arithmetic varied greatly between different computer manufacturers This has been lessoned significantly with the development of the IEEE standard for computer floating-point arithmetic. 29

30 Effects of computer hardware For large scale problems, especially in numerical linear algebra, it is important to know how the elements of a matrix A or a vector x are stored in memory. Knowing this can lead to much faster transfer of numbers from the memory into the arithmetic registers of the computer, thus leading to faster programs. A somewhat related topic is that of pipelining. This is a widely used technique whereby the execution of computer operations are overlapped, leading to faster execution. Machines with the same basic clock speed can have very different program execution times due to differences in pipelining and differences in the way memory is accessed. 30

31 Common Pers. in NA Numerical analysis is concerned with all aspects of the numerical solution of a problem, from the theoretical development and understanding of numerical methods to their practical implementation as reliable and efficient computer programs. Most numerical analysts specialize in small subareas, but they share some common concerns, perspectives, and mathematical methods of analysis. These include the following: 31

32 Common Pers. in NA When presented with a problem that cannot be solved directly, then replace it with a nearby problem which can be solved more easily. Examples are the use of interpolation in developing numerical integration methods and rootfinding methods. 32

33 Common Pers. in NA There is widespread use of results of linear algebra, real analysis, and functional analysis (with its simplifying notation of norms, vector spaces, and operators). 33

34 Common Pers. in NA There is a fundamental concern with error, its size, and its analytic form. When approximating a problem, it is prudent to understand the nature of the error in the computed solution. Moreover, understanding the form of the error allows creation of extrapolation processes to improve the convergence behavior of the numerical method. 34

35 Common Pers. in NA Stability is a concept referring to the sensitivity of the solution of a problem to small changes in the data or the parameters of the problem. 35

36 36

37 Common Pers. in NA Numerical analysts are very interested in the effects of using finite precision computer arithmetic. This is especially important in numerical linear algebra, as large problems contain many rounding errors. 37

38 Common Pers. in NA Numerical analysts are generally interested in measuring the efficiency of algorithms. What is the cost of a particular algorithm. For example, the use of Gaussian elimination to solve a linear system Ax = b containing n equations will require approximately 2/3n 3 arithmetic operations. How does this compare with other numerical methods for solving this problem? 38

39 Modern App. & Com. Software Sophisticated numerical analysis software is being embedded in popular software, e.g. spreadsheet programs, allowing many people to perform modelling even when they are unaware of the mathematics involved. It requires designing problem solving environments (PSE). The PSE for a given problem area is usually based on excellent theoretical mathematical models, made available to the user through a convenient graphical user interface, e.g., computer aided design of structures, and atmospheric modelling, investment and banking. 39