Orders of Growth. Also, f o(1) means f is little-oh of the constant function g(x) = 1. Similarly, f o(x n )

Size: px

Start display at page:

Download "Orders of Growth. Also, f o(1) means f is little-oh of the constant function g(x) = 1. Similarly, f o(x n )"

Charla Joseph
5 years ago
Views:

1 Orders of Growth In this handout, if it makes sense for functions to have domain R n and range R m, assume they do. But sometimes they need to be real functions of a real variable, in order for claims to make sense, or for proofs to be correct, or to avoid considerable hassle. Use your judgment. Definition. f o(g) (spoken f is little oh of g ) ɛ>0 δ>0 [ x δ= f(x) ɛ g(x) ]. (1) Furthermore, we say f(x) =g(x)+o(h(x)), or just f = g + o(h), iff f(x) g(x) o(h(x)). Note 1. This definition, and similar definitions below for things like O(g), are about what happens as x 0. Typically both f and g go to 0, and the question is: does one go to 0 much faster than the other. Thus orders of decrease might be a better name for this subject. However, one can study change as x goes to other points. See Problem 50 for an introduction to the case where x. In that case, both functions of interest typically go to, and the question is: which one grows faster? Thus orders of growth is the appropriate phrase in that case, and the name has stuck. Note 2. The most common g(x) isg(x)=x. So f o(x) means f o(g) where g(x) =x. Also, f o(1) means f is little-oh of the constant function g(x) = 1. Similarly, f o(x n ) means g(x) =x n. Note 3. The traditional notation for f is little oh of g is f = o(g). But o(g) really stands for the set of all functions that are little oh of g, so I prefer to use. Similarly, one should really write f g + o(h), meaning f is in the set of functions obtained by adding g to each function in o(h). However, when one writes f = g + o(h), one is saying f is of a particular form the form of g plus a little extra. So I do prefer = in this case. 1. The definition of f o(g) given in (1) is not what one always sees. Sometimes one sees f(x) lim =0, (2) x 0 g(x) or f(x) lim =0. (3) x 0 g(x) Why is (3) better than (2) and (1) better than (3)? 2. Show: In (1), it is equivalent to write x <δinstead of x δ, but it is not equivalent to write f(x) <ɛ g(x) instead of f(x) ɛ g(x). Hint: What if g(x) = 0 for some x with x δ? Very often, g(0) = 0. Note: In ɛ-δ definitions of continuity, either < or can be used with either Greek letter; but not here. 3. Prove: If f o(g) and g o(h), then f o(h). 4. Prove or disprove: if f o(g), then there exists h(x) for which f o(h) and h o(g). 2/6/98, revised June 4, 1998

2 5. Suppose f o(g) and h f, meaning h(x) f(x) for all x. Then it need not be true that h o(g). a) Give an example. b) Make a small change in the claim so that it is true. 6. Prove: if a + bx + o(x) =c+dx + o(x), then a = c and b = d. 7. Find a quadratic q(x) for which you can prove e x = q(x) +o(x 2 ). Hint: The remainder theorem for Taylor polynomials will help. 8. If g, h o(f), which of the following are in o(f)? (Give proofs). a) g ± h b) cg (c a constant) c) g(x)h(x), assuming g, h are real-valued functions d) g(x) h(x) (dot product) 9. For this problem, for simplicity assume functions have real inputs and outputs, not vectors. a) Interpret and prove: o(g) + o(g) = o(g) b) Interpret and disprove o(g) o(g) = o(g). c) Interpret and prove o(x) o(x) = o(x). d) Interpret and prove o(o(x)) = o(x). e) Since, in the oh-trade, f = o(g) means f o(g), a reasonable interpretation of o(g)+o(g) = o(g) is the set statement o(g) + o(g) o(g). However, another interpretation is that = really means =, of sets. i) If you used the interpretation in part a), go back and prove or disprove o(g)+o(g) = o(g) with the equals interpretation. ii) Go back and prove or disprove o(x) o(x) = o(x) with the equals interpretation. 10. Let f(x) be any function continuous at x = 0. Interpret, then prove or disprove, f(x) o(g) =o(g). 11. If f(x)= 2 + x+ o(x), and g(x) = 5 3x+ o(x), what can you say about a) f(x)+g(x); b) f(x)g(x)? Hint: Add the equations, then multiply them. 12. Let f(x)=2+3x x 2 +o(x 2 ), g(x) =2x+x 2 +o(x 2 ). Express each of the following in the form p(x)+o(x n ), where p(x) is a polynomial of degree n and n is as large as you can justify (and thus o(x n ) is as small as possible as x 0). a) f(x)+g(x) 2

3 b) f(x)g(x) c) [g(x)] 2 d) f(g(x)) e) Why insist that p(x) have degree n? Might not the right answer be something like x 3x 2 +2x 3 +o(x 2 )? 13. Do Problem 12 again, but change g(x) to 2x+ o(x). 14. Let f(x) = 1 + x + x 2 + o(x 2 ), g(x) = 1 x + x 2 + o(x 2 ). Show that f(x)/g(x) = 1+2x+2x 2 +o(x 2 ). Hint: do long division. You ve probably never done long division with ohs before. Why is it legitimate? 15. If f(x)=2+x x 2 +o(x 2 ), is f(2x)=2+2x 4x 2 +o(x 2 )? 16. a) If f(x)=o(x), is f(2x) =o(x)? b) If f(x)=o(x n ), is f(2x) =o(x n )? c) If f(x)=o(g(x)), is f(2x) =o(g(x))? 17. Give a counterexample to the claim o(o(1)) = o(x). 18. If f(x)=o(x n ) for some n>1, one would hope that f (k) (0) would be 0 for k =1,2,...,n. However, all that follows necessarily is that f (0) = 0. Specifically, a) Show that f (0)=0. b) Show that f (0) need not even exist. 19. Suppose g i o(f i ) for i =1,2. Is it necessarily true that a) g 1 + g 2 o(f 1 +f 2 )? b) g 1 g 2 o(f 1 f 2 ) if the f s and g s are real-valued functions? c) g 1 g 2 o(f 1 f 2 )? (dot product) d) g 1 /g 2 o(f 1 /f 2 )? 20. If f,g are real-valued functions, define the function f g by (f g)(x) = max{f(x),g(x)}. Now assume f i = o(g i ) for i =1,2. Prove or disprove a) f 1 + f 2 = o( g 1 + g 2 ) b) f 1 f 2 = o( g 1 g 2 ) Definition. f O(g) (spoken f is big oh of g ) M>0,δ>0 [ x δ= f(x) M g(x) ]. (4) Furthermore, we say f(x) = g(x) + O(h(x)) iff f(x) g(x) O(h(x)). 3

4 21. State some other forms of definition (4) that aren t as good. Why not? (See Problem 1.) 22. What does it mean to say f O(1)? That is, are there other standard mathematical words or phrases you can use to describe what s going on? 23. If f O(g), is g O(f)? 24. If f o(g), is f O(g)? 25. Interpret and prove: O(f) + o(f) = O(f). 26. a) Explain why every polynomial p(x) satisfies for some constants a, b. b) Explain why every polynomial p(x) satisfies p(x) =a+bx + o(x) (5) p(x) =a+bx + O(x 2 ) (6) for some constants a, b. c) Which statement above, (5) or (6), is stronger, and what does stronger mean? This part is supposed to tell you something about the relative merits of describing remainders using o and O. 27. Prove If p(x) is a polynomial with lowest degree k, then p(x) O(x k ) and p(x) o(x k+1 ). 28. Let f(x)=2+3x x 2 +O(x 3 ), g(x) =2x+x 2 +O(x 3 ). Express each of the following in the form p(x)+o(x n ), where p(x) is a polynomial (of as high a degree as is useful to write) and n is as large as you can justify (and thus O(x n ) is as small as possible as x 0). a) f(x)+g(x) b) f(x)g(x) c) [g(x)] 2 d) f(g(x)) 29. Do Problems a) 3, b) 8, and c) 19 with O in place of o. 30. Many previous problems have been on the question: if f,g are in a certain relation to h (say f,g O(h)), are various functions made from f and g also in the same relation to h? But we can also ask if these functions made from f and g are in a different relation to h, or in a relation to something made from h. Interpret and prove or disprove: a) o(x) o(x) =o(x 2 ), where x is real-valued. b) o(g) o(g) =o(g 2 ), where g(x) is real-valued. c) o(g) O(g) =o(g 2 ), where g(x) is real-valued. d) o(h) O(h) =o( h 2 ), where h is vector-valued. 4

5 e) O(h) O(h) =o(h). (If this claim is not true for all h, can you say which h this is true for?) 31. Prove: o(o(h)) = o(h), but first state carefully what this claim means. 32. If f o(g), is f(x) O(xg(x))? 33. Prove O ( xo(x 2 ) ) = O(x 3 ). This can be proved quickly with certain results earlier on this handout. 34. Find a function f such that f(x) o(x 1+ɛ ) for all ɛ>0, yet f(x) / O(x). Definition. Wesayfand g are the same order, and write f Θ(g), iff δ>0,l>0,u >0 L g(x) f(x) U g(x) whenever x δ. (7) (L and U stand for lower and upper bounds.) 35. If we say of the same order, it better be the case that f Θ(g) = g Θ(f). Prove it. 36. Show that, for f to be the same order as g, it is sufficient that lim f(x)/g(x) exist and not x 0 be Show that Θ is an equivalence relation, that is: i) Every function is the same order as itself; ii) If f is the same order as g, then g is the same order as f; and iii) If f Θ(g) and g Θ(h), then f Θ(h). The functions in Θ(f) are called the order equivalence class of f. 38. Suppose g i Θ(f i ) for i =1,2. Is a) g 1 g 2 Θ(f 1 f 2 ) if the f s and g s are real-valued functions? b) g 1 /g 2 Θ(f 1 /f 2 )? 39. Prove: o(x 2 ) = o(x). (8) Θ(x) State a more general result of which (8) is a special case. Definition. Wesayfand g are asymptotic, and write f g, iff lim f(x)/g(x) =1. x Write a better definition of f g, one that works even if g(x) is repeatedly 0 near the origin. 41. Show that x 2 + x 3 x Show that 2 x 1 x but 2 x 1 Θ(x). 43. Prove that f g = g f using your definition in Problem 40. 5

6 44. Prove that another equivalent definition of f g is that (f g) o(g). 45. Theorem. For vector functions, (f g) = f g. Prove this two ways: a) Using the definition of derivative and limit theorems; b) Restating the definition of derivative using little oh and then using little oh facts. 46. a) Challenge: do a traditional limits proof of the multivariable product rule (for (fg) where f,g:r n R). No o(h) s. b) Figure out an o(h) proof of the multivariate chain rule. Prove any o and O facts you need for the proof that aren t included in the problems above. Order of functions. We say Θ(g) Θ(f) (in words, that the order of g is smaller than the order of f) iff g o(f). 47. Rank the order of the following functions from smallest to largest. You may assume x 0 from the positive side only. x, x 2, x 3, 1/x, e x, e 1/x, e 1/x, ln x, 1/ ln x, x ln x, x 2 / ln x, x(ln x) The definition of Θ(g) Θ(f) suggests that every function in the equivalence class of Θ(g) is smaller than every function in Θ(f). That is, if g o(f), g Θ(g), and f Θ(f), then g o(f ). Prove that this is true. 49. Is there a trichotomy law for the way there is for < and numbers? That is, for any two functions f,g, must exactly one of the following be true: f Θ(g), Θ(f) Θ(g), or Θ(g) Θ(f)? 50. So far, we have considered the relative behavior of functions only as x 0. However, the limit point could be any number or vector, or even. Rewrite the definitions of o, O, Θ, when the limit is. (The case where the limit is is the main case of interest in discrete math and computer science, where one uses o, O, Θ, to study the efficiency of algorithms as the input gets larger and larger. The case where the limit is finite is the main case in calculus and real analysis.) 51. Rank the order of the following functions from smallest to largest (as x ): x, x 2, x 3, 1/x, e x, 2 x, ln x, log 10 x, x ln x, x 2 / ln x, x(ln x) Let g(n) = n k=1 k. Find a power p so that (as n )g Θ(np ). 53. Let H(n) = n k=1 1 k. Prove that (as n ) H(n) ln n. 6

. As x gets really large, the last terms drops off and f(x) ½x

. As x gets really large, the last terms drops off and f(x) ½x Pre-AP Algebra 2 Unit 8 -Lesson 3 End behavior of rational functions Objectives: Students will be able to: Determine end behavior by dividing and seeing what terms drop out as x Know that there will be