WELCOME! Lecture 14: Factor Analysis, part I Måns Thulin

Size: px

Start display at page:

Download "WELCOME! Lecture 14: Factor Analysis, part I Måns Thulin"

Abigayle Brittney Parks
5 years ago
Views:

1 Quantitative methods II WELCOME! Lecture 14: Factor Analysis, part I Måns Thulin

2 The first factor analysis C. Spearman (1904). General intelligence, objectively determined and measured. The American Journal of Psychology. Children's performance in mathematics (Y 1 ), French (Y 2 ) and English (Y 3 ) was measured. Regression model: Y 1 =λ 1 f+e 1 Y 2 =λ 2 f+e 2 Y 3 =λ 3 f+e 3 Where f is is an underlying "common factor" ("general ability").

3 Factor analysis In many areas it is often not possible to measure directly the concepts/constructs of primary interest. Two obvious examples are intelligence and social class. In such cases the researcher has to examine the concepts indirectly by collecting information on variables which can be measured, and which are considered indicators of the concepts of real interest. The immeasurable concepts are called latent variables, factors, or constructs

4 Examples of latent variables (factors) Psychology - Intelligence, Depression, Self-esteem, Drinking behavior, Feelings Sociology - Social class, Occupational aspiration, Ambition, Discrimination Economics - Economic expectations, Empowerment of women Education - Academic performance, Science achievement Business - Customer satisfaction, Customer Loyalty Political Science - Industrial development, Political Efficacy

5 Exploratory and confirmatory factor analysis Factor analysis can be used in at least two ways: Exploratory factor analysis: an exploratory technique, to investigate the relationships between observable indicator variables and latent variables (factors, constructs). Confirmatory factor analysis: a technique for testing a specific concept structure in which particular indicator variables relate to particular latent variables.

6 Exploratory factor analysis Purpose of using exploratory factor analysis: Develop instruments to measure the concepts of interest (e.g. questionnaires measuring intelligence). Reduce the number of variables studied. Summarize and describe the data. Generate hypotheses and construct models of abstract quantities. To perform a factor analysis, we need to study correlations.

7 Relationship between two quantitative variables We are often interested in the relationship between two or several quantitative variables. A relationship can have a specific shape, and be of a specific strength.

8 Relationship between two quantitative variables To analyze the relationship between two quantitative variables start with visualizing it, using a scatterplot. Since we study two variables at the same time, every individual has two values. We can describe each individual s combination of these two values as a point in the scatterplot.

9 Variable 2 Variable 2 Scatter plots Variable 1 Variable 1 The linear pattern (trend) indicates a relationship A round cloud shows no indication of a relationship 9

10 Correlation The Pearson correlation coefficient (r ) measures the direction and strength of the linear relationship between two variables X andy The correlation is always a number between -1 and 1-1 r 1

11 Correlation If there is a strong positive linear relationship between X and Y, the value of the correlation coefficient (r ) is close to 1. If there is a strong negative linear relationship between X and Y, the value of the correlation coefficient (r ) is close to -1. If there is no linear relationship at all between X and Y, the value of the correlation coefficient (r ) is close to 0. Y Y Y r = 1 r = -1 r = 0 X X X

12 Examples of scatterplots with different correlations Y Y Y r = -1 X r = -.6 X r = 0 X Y Y Y r = +1 X r = +.3 X r = 0 12 X

13 Non-linear relationships Pearson s r close to 0 Y Y Y X X X The Spearman correlation coefficient (r s ) measures the strengh of a relationship that is not necessarily linear

14 Pearson s correlation coefficient Measures the strength of a linear relationship (r ) between two variables X and Y. Assumptions: Random samples Both variables are numerical (quantitative) No extreme values Disadvantages: Very large samples often give a significant correlation (low P-value) Difficult to show relationships for small samples Does not describe curved relationships, only linear

15 Spearman s correlation coefficient Measures the association (r s ), not necessarily linear, between two variables X andy Assumptions: Random sample(s) The variables can at least be ranked (ordered) Advantages: Reflects also curved relationships

16 Correlation matrix When there are more than two variables, correlations are usually displayed in a matrix. Spearman s 1904 study of intelligence: children's performance in mathematics (Y 1 ), French (Y 2 ) and English (Y 3 ) was measured. Correlation matrix: Y 1 Y 2 Y 3 Correlation between Y 1 and Y 2 Correlation between Y 3 and Y 3. Always 1! Y 1 Y2 Y 3 Correlation between Y 1 and Y 3 Correlation between Y 2 and Y 3

17 Examples of latent variables (factors) Psychology - Intelligence, Depression, Self-esteem, Drinking behavior, Feelings Sociology - Social class, Occupational aspiration, Ambition, Discrimination Economics - Economic expectations, empowerment of women Education - Academic performance, Science achievement Business - Customer satisfaction, Customer Loyalty Political Science - Industrial development, Political Efficacy

18 The basic factor analysis model Assume that we have p observed variables; x 1, x 2,, x p. Example: 10 work related questions in a questionnaire. The observed variables are assumed to be linked to a smaller number of unobserved latent variables, 1, 2,, k, where k < p.

19 The basic factor analysis model 1 2 Factors (unobservable latent variables/constructs) Loadings (correlations) x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 Observed variables (indicators) Error terms (unique factors)

20 Examples of factor analysis models One-factor model: 1 x 1 x 2 x 3 x 4 x 5 x

21 Examples of factor analysis models Two-factor model: x 1 x 2 x 3 x 4 x 5 x

22 The basic factor analysis model The following regression model is assumed: x x x k k k k 2 p.... p1 1 p2 2 pk k p Observed variables (indicators) Loadings (weights) Unobserved latent variables (factors/concepts) Error term

23 Example: Students ability and aspiration Calsyn and Kenny* collected data on the six following variables for 556 eighth-grade students: x 1 : self-concept of ability x 2 : perceived parental evaluation x 3 : perceived teacher evaluation x 4 : perceived friend s evaluation x 5 : educational aspiration x 6 : college plans *Calsyn, J.R. and Kenny, D.A. (1977). Self-concept of ability and perceived evaluation of others. Cause or effect of academic achievement? Journal of Educational Psychology, 69,

24 Example: Students ability and aspiration Ability Aspiration The observed variables could be indicators for both latent factors x 1 x 2 x 3 x 4 x 5 x

25 Example: Students ability and aspiration x x ( 12 2) ( 22 2) 2 Where 1 = ability 2 = aspiration x x x x ( 32 2 ) ( 42 2) 4 5 ( 51 1) ( 61 1)

26 Example: AIDS patients reactions Aids patients reactions to their physicians have been investigated using a survey.* 14 items were included in the survey questionnaire, which measure the patient attitudes about physician personality, behavior, competence, and prescribed treatment. The patients rated each item using a so-called Likert scale from 1 to 5 *From the book Applied Multivariate Data Analysis by Everitt & Dunn

27 Example: AIDS patients reactions Items: 1) My doctor treats me in a friendly manner. 2) I have some doubts about the ability of my doctor. 3) My doctor seems cold and impersonal. 4) My doctor does his/her best to keep me from worrying. 5) My doctor examines me as carefully as necessary. 6) My doctor should treat me with more respect. 7) I have some doubt about the treatment suggested by my doctor. 8) My doctor seems very competent and well trained.

28 Example: AIDS patients reactions Items, cont d: 9) My doctor seems to have a genuine interest in me as a person. 10) My doctor leaves me with many unanswered questions about my condition and its treatment. 11) My doctor uses words that I do not understand. 12) I have a great deal of confidence in my doctor. 13) I feel I can tell my doctor about very personal problems. 14) I do not feel free to ask my doctor questions.

29 Example: AIDS patients reactions Competence Personality Behaviour Treatment x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 x 9 x 10 x 11 x 12 x 13 x

30 Example: AIDS patients reactions x x x Where 1 = personality 2 = behavior 3 = competence 4 = prescribed treatment

31 Factors can be unknown In exploratory factor analysis, the latent factors are typically unknown. A set of correlated observed variables are being analyzed without necessarily knowing in advance either the number of factors that are required to explain their interrelationships or their meaning or labeling. Depending on the model finally chosen, the factors are named according to the indicators to which factor is related.

32 Confirmatory Factor Analysis Confirmatory factor analysis postulates certain relationships among the observed variables and the latent factors assuming a pre-specified pattern for the model parameters. This type of analysis is mainly used for testing a hypothesis arising from theory (hypotheses about factor loadings, usually that some are zero). Therefore, the number of latent factors and the observed indicator variables that will be used to measure each latent factor are known/specified in advance.

Introduction To Confirmatory Factor Analysis and Item Response Theory

Introduction To Confirmatory Factor Analysis and Item Response Theory Lecture 23 May 3, 2005 Applied Regression Analysis Lecture #23-5/3/2005 Slide 1 of 21 Today s Lecture Confirmatory Factor Analysis.