Intro to Stats Lecture 11

Transcription

1 Outliers and influential points Intro to Stats Lecture 11 Collect data this week! Midterm is coming! Terms X outliers: observations outlying the overall pattern of the X- variable Y outliers: observations outlying the overall pattern of the Y- variable (Model) outliers: observations outlying the overall pattern of the X-Y association. Outliers can be detected in an residual plot, (only sometime, not always.) Influential points (observations): observations that have substantial influence on the model fitted. Removing such observations will result in quite different models (regression lines). REMOVED 1

2 Regression with summary values Fitting a regression model for summary values (such as average) against an explanatory variable. Example: average height versus age. Reduced variation due to averaging. Thus the model is for the averaged values NOT the original variable. linear transformation Change the units of measurement price ($) = * size (squared foot) Transformation: price($) = (1/8) price(rmb) size (squared foot) 9 size (squared meter) Model equation on the new scale: price($) = (1/8) price(rmb) = size (sq ft) = (9 size (sq m)) = size (sq m) price(rmb) = size (sq m) power transformation Straighten the line (gallon per 100 miles) = weight (lb) Transformation: (gallon per 100 miles) = 100/MPG Model on original scale. (gallon per 100 miles ) = 100/MPG = weight 100 MPG = weight Log transformation We consider the log based-10. Examples: log(yield of leaves) = 2+1.7log(height of a tree) Why transform? Straighten the line Stabilize the residual variation Required only for homework problems Requirement for transformation Rewrite regression line for Linear transformation Power transformation For linear transformation Know the effects of the linear transformation on the measures of center (mean or median) and the measures of spread (standard deviation or IQR). 2

3 Random phenomenon Uncertain outcomes: what will be the highest temperature tomorrow? Trend: we know it is unlikely to be in the 80 s since it is October, based on weather data from the past. Unpredictable outcomes plus long-term trends random phenomenon that can be studied using probability Probability The long-run relative frequency. True proportion of a given outcome in a large number of independent random trials. Remember? Trial means one observation of a random phenomenon. Independent? The trials are independent of each other when the outcome of a trial won t affect the outcome of the other trials. Law of Large Numbers Probability: a mathematical (idealized) quantity, which describes long-term trend. Consider (completely) random number between 1 to 10. (See the handout) The mechanism of the simulation should have given a same chance to each number probability. Long-term relative frequency of each outcome gets closer to the true probability (LLN) Don t let your instinct fool you! Beware of the law of small numbers If I flip a coin 10 times, I should have exactly 5 heads and 5 tails. Good luck with that! Exact probability rarely shows up in a small number of trials. Beware of the law of averages I have gotten 5 heads out 5 so far, I think I am about to have a tail coming up. Every trial (if done correctly) should be independent of the outcome of previous trials. Assign probability to a random phenomenon What does one need to describe a random phenomenon? All possible outcomes (outcomes don t overlap.) Chance or probability of observing each of the outcomes Sample space: the collection of all possible outcomes Event: is an outcome or a set of outcomes out of the sample space. A subset of the sample space. Probability of an event: sum of the probability of the outcome(s) included in this given event. Observing an event means you observe one of the outcomes that belong to an event. Assign probability (cont d) Pop quiz: Sample space for the random phenomenon that a fair coin is tossed. Sample space for the random phenomenon that a student is picked at random out of this class room. Event A: the picked student is a CC student Event B: the picked student is a girl Event C: the picked student is a girl from CC 3

4 Assign probability (cont d) Notations: Sample space S Events letters at the beginning of the alphabet Probability of event A Prob(A) or p(a) Assign probability (cont d) Probability rules For any event, 0 Prob(A) 1. (Hint: relative frequency!) Prob(S)=1. Reason: If your sample space has all possible outcomes of a random phenomenon, you should observe at least one of them during every trial. P(A C )=1-P(A); Here A c is the complement of A. If A and B don t overlap, P(A or B) = P(A) + P(B) If A and B are independent, P(A and B) = P(A)P(B) What did I mean by the complement of A? Event A: the picked student is a CC students Event A C : the picked student is NOT a CC student. (the not-a event) A C has all the outcomes that is not in A. A C plus A together contains all outcomes in the sample space they complete each other. What did I mean by A or B? Event D: the picked student is a CC student or a girl. Event D is the UNION of two events A (CC) and B (girl). Yes, she can be both but doesn t have to! Sometime, there is no overlap. For example, CC student or BC student. What did I mean by A and B? Event C: the picked student is a CC girl. Event C is an INTERSECTION of event A (CC) and event B (girl). She has to satisfy BOTH requirements of the event C s definition. Sometime, when A and B do not overlap, A and B returns nothing: the empty set. Assign Probability to an event Given probability of all possible outcomes Toss a loaded die (see board) Under the assumption of equally likely outcomes: # of outcomes in A PA ( ) = total # of outcomes 4

5 Example 1: chapter 14, problem 11 A consumer organization estimates that over a 1-year period 17% of cars will need to be repaired once, 7% will need repairs twice, and 4% will require three or more repairs. What is the probability model here? What is the probability that a car chosen at random will need No repair? No more than one repair? Example 2 A manufacturer claims that 99% of his light bulbs last longer than 2000 hours. I bought 2 such bulbs. What is the probability that they both last less than 2000 hours? Reading Chapter 14 Don t say I didn t remind you. You should start reading for the midterm. 5