Statistics A Brief Visit. Lulu Kang, MATH 100

Transcription

1 Statistics A Brief Visit Lulu Kang, MATH 00

2 What is Statistics Many people would think statistics is the study of data. True, but not entirely true.

3 What is Statistics Statistics is the study of the collection, organization, analysis, interpretation, and presentation of data. (Dodge, Y. (006) The Oxford Dictionary of Statistical Terms. ) Statistics is everywhere in our life.

4 Data Collection There are so many ways to collect data Manufacturing: sensor, scanner, high speed camera, Medical: any exam including blood test, X- ray, fmri, Social: cell phone, any online record, bank account, survey,

5 Data Collection needs to be Smart Some data collection is cheep. Some data collection is really expensive. To make sure the collected data are truly useful with the limited budget, we need to use Design of Survey Design of Experiments.

6 Design of Survey How to construct effective surveys: a toy example. Example Balanced: Very Poor Poor Average Good Excellent 4 5 Example Unbalanced: Poor Average Good Very Good Excellent 4 5

7 Design of Experiments What is the best recipe? Oven Temperature Sugar Flour Eggs

8 Data Analysis Analysis of data is a process of inspecting, cleaning, transforming, and modeling data with the goal of discovering useful information, suggesting conclusions, and supporting decision-making.

9 Data Analysis There are so many data analysis techniques. Simplest one: linear regression. The higher the reaction temperature, the larger the yield. Thus to increase yield, the manufacturer needs to increase the temperature as high as possible.

10 Data Analysis Time Series Model: y-hotel sales v.s. months y Time: Monthes Periodic: similar pattern every year, summer travels the most in the summer (the highest peak), and some travel peak in holidays. Global: increasing trend. One of the reason might be economic growth, and improving of living standard.

11 Data Analysis Graphical Model History(of(Smoking((HS)( HS( Chronic(Bronchi9s((CB)( CB( LC( Lung(Cancer((LC)( F( W L( Fa9gue((F)( Weight(Loss((WL)( Causal Study: what can be the cause of lung cancer, and what are its symptoms

12 Data Analysis Classification FIGURE 4.. The left plot shows some data from three classes, with linear decision boundaries found by linear discriminant analysis. The right plot shows quadratic decision boundaries. These were obtained by finding linear boundaries in the five-dimensional space X,X,X X,X,X. Linear inequalities in this space are quadratic inequalities in the original space. Ever wonder why your Gmail can tell spam from non-spam s? Or how they can tag your into: Primary, Social, Promotions, etc?

13 Data Visualization How to interpret the data? Data Analysis Data Visualization: visual representation of data information that has been abstracted in some schematic form, including attributes or variables for the units of information. (Michael Friendly, 008) Effective visualization helps users in analyzing and reasoning about data and evidence.

14 Common Techniques Network Example: my LinkedIn network linkedin-inmaps/

15 Common Techniques Bar chart Example: SAT Scores and Family Income

16 Common Techniques Stream graph Example: Box Office Receipts More examples

17 Common Techniques Bubble Example: Facebook IPO

18 Others: ancient Napoleon s Invasion of Russia This 86 diagram by Charles Joseph Minard illustrates the advance and retreat of Napoleon's army in Russia from 8 to 8. The thickness of the line indicates the size of the army. From left to right, the thick line on top shows the army crossing the Neman River with 4,000 men, advancing into Russian territory and stopping in Moscow with just 00,000 men. From right to left, the lower line shows the army returning west, including the disastrous crossing of the Berezina River. Only a small fraction of Napoleon's army, approximately 0,000 men, survived. The lower portion of the graph shows the temperature during the army's retreat, in degrees below freezing on the Réaumur scale. by Charles Joseph Minard, 986

19 Others: modern Equal Population Mapper The iphone Economy

20 How Statistics can do to you? For Today s Graduate, Just One Word: Statisitcs NYTimes I keep saying that the sexy job in the next 0 years will be statisticians, said Hal Varian, chief economist at Google. And I m not kidding. ( 06stats.html)

21 To begin with MATH 474: probability and statistics MATH 476: Statistics MATH 484: Regression and Forecasting MATH 569: Statistical Learning MATH 574: Bayesian Computational Statistics