Measurement Issues in Business Research

Transcription

1 1 Measurement Issues in Business Research Kenneth S. Law Department of Management The Chinese University of Hong Kong 香港中文大学管理系

2 2 Psychological Measurement Clarify some terms Four types of scales Reliability 信度 Validity 效度 Measurement scales Effect of reliability on validity How to define a construct

3 3 Psychological Measurement Clarify some terms Four types of scales Rlibili Reliability 信度 Validity 效度 Measurement scales Effect of reliability on validity How to define a construct

4 4 Psychological construct A construct ( 构念 ) is a concept deliberately and consciously invented or adopted for a specific scientific ifi purpose. For example: Temperature Intelligence Job satisfaction Organizational culture Job embeddedness

5 5 Construct and measures o C Celsius Temperature concrete abstract observable bl unobservable Feeling

6 6 Construct and measures Temperature abstract 构念 Construct unobservable Measure concrete 测量 observable bl

7 7 Construct and measures Volume Measure

8 8 Construct and measures Turnover Intention 构念 Construct Disagree Agree I always think about leaving Measure 测量

9 9 Construct and measures Job Satisfaction 不同意 同意 Measure 1. 我很满意我的工作 我工作非常开心 我对工作感到很满足

10 10 Construct and measures Firm performance $ 构念 Construct ROI Productivity/Employee Measure 测量 ROE

11 11 Psychological construct Reality Abstract theorization Measures Moving fast/slow Vl Velocity Km/hour Fast/slow learner Happy/unhappy about job Different norms in different organizations One s tie to the job Intelligence Job satisfaction Organizational culture Job embeddedness IQ score OCQ Value survey 42-item measure

12 12 Question to think of? Does organizational commitment really exist in this world? Does gravitational force really exist in this world? Does magnetic field really exist in this world? Does job satisfaction really exist in this world?

13 13 Predictor and criterion Organizational commitment Turnover Intention Independent variable ( 自变量 ) Dependent variable ( 他变量 ) Predictor Ci Criterion i

14 14 Some characteristics of constructs 1. Constructs are being constructed; 2. Constructs are abstract and not directly observable; 3. Constructs are developed for building models and theories; 4. Constructs should be clearly defined; 5. Constructs need to be operationalized (measured). E = m c 2 How to measure that? LMX OCB

15 15 A unidimensional construct Measurement items observable Construct abstract I often think about quitting. x 1 I would look for a new job next year. x 2 Turnover Intention I would prefer to stay if I have a choice. x 3

16 16 Dimensions vs. items The critical aspects of job embeddedness are (1) the extent to which people have links to other people or activities; (2) the extent to which their jobs and communities are similar to or fit with other aspects in their life spaces, and, (3) the ease with which links can be broken what they would give up if they left, especially if they had to physically move to other cities or homes. (p.1104) Links are characterized as formal or informal connections between a person and institutions or other people. (p.1104) Fit is defined as the employee s perceived compatibility or comfort with an organization and with his or her environment. (p.1104) Sacrifice captures the perceived cost of material or psychological benefits that may be forfeited by leaving a job. (p.1105) Mitchell, T.R., Holtom, B.C., Lee, T.W., & Erez, M. (2001). Why people stay: Using job embeddedness to predict voluntary turnover, Academcy of Management Journal, 44(6),

17 17 A multidimensional construct Items Dimensions Construct Are you currently married? How long have you been in your present position? Links Internal External Fit Job Embeddedness Turnover Internal External Sacrifice

18 18 Formative vs. Reflective indicators Income Relax Parent s income Socioeconomic status Life satisfaction Happy Size of apartment Positive Formative or causal indicators Reflective or effect indicators 18

19 19 Constructs must be clearly defined For example, I may propose a new construct called organizational relationship, which describes the overall relationship of an employee with his/her organization. What does organizational relationship means? What kind of relationship are we talking about? organizational relationship may be highly related to many established constructs such as perceived organizational support (POS), leader-member exchange (LMX), organizational commitment (OC) and organizational identification (OI), trust or turnover intention. Therefore, it seems that t organizational relationship may not be a good construct, because we do not know what you are really talking about. At the end, the construct may be relating to many variables but the only reason is that it is not well-defined.

20 20 Poorly defined construct y 1 X r xy1 =.76 Organizational i Relationship r xy2 =.82 y 2

21 21 How to measure constructs 1. Ui Using idi indicators e.g., ROI as an indicator of firm performance Single indicator vs. multiple indicators 2. Using measurement scales/items e.g., MSQ as a measure of job satisfaction 3. Using proxies e.g., Academic performance as a proxy of general mental ability

22 22 Measurement Scales Disagree Agree 1. 通常我能知道自己会有某些感受的原因 我很了解自己的情绪 我真的能明白自己的感受 我常常知道自己为甚么觉得开心或不高兴 遇到困难时, 我能控制自己的脾气 我很能控制自己的情绪 当我愤怒时, 我通常能在很短的时间内冷静下来 我对自己的情绪有很强的控制能力 我通常能为自己制订目标并尽量完成这些目标 我经常告诉自己是一个有能力的人 我是一个能鼓励自己的人 构念 : 情绪智能 Emotional Intelligence Measurement Scale 12. 我经常鼓励自己要做到最好 测量 : 量表 13. 我通常能从朋友的行为中猜到他们的情绪 我观察别人情绪的能力很强 我能很敏锐地洞悉别人的感受和情绪 我很了解身边的人的情绪

23 23 Measurement Scale A scale measuring personal integrity Why scale? 同意没意见不同意 只是不犯法, 歪曲法律並无不可 在学时我不只一次被校长处罚 大部分人会为了超越他人而说谎 偶尔一次说谎是人之常情 规则是为人而设立的, 被人打破也没什么大问题 把有价值的东西乱放的人, 应该与偷窃这些 东西的人同受责备 Use of scales to reduce error (for reliability and validity concerns).

24 24 Business Research (Unseen Constructs) (Predictor) Organizational commitment (Criterion) Turnover Intention 我对我的组织非常忠诚 我常常想到离开公司 ( Item1 ) ( Item 7 ) 我非常投入我的工作 我正在外面找工作 ( Item 2 ) ( Item 8 )

25 25 Relationship between ee constructs and indicators x t y t Construct 构念 不可观察 Commitment r xt t y t Turnover intention 抽象?????? Indicator 指标 可观察 实在 I am proud of working in this company. x r xy I always think of leaving. y 6-item 3-item Meyer, Allen & Smith (1993) Cammann et al. (1979)

26 26 Measurement is important 1. Construct A and B correlate positively. (theory) 2. X is a measure of construct A. 3. Y is a measure of construct B. 4. X and Y correlate positively. If, in reality, r xy = 0 Either X is not a good measure of A (measurement). Or Y is not a good measure of B (measurement). Or Construct A & B are not correlated (theory).

27 27 Psychological Measurement Clarify some terms Four types of scales Rlibili Reliability 信度 Validity 效度 Measurement scales Effect of reliability on validity How to define a construct

28 28 Four types of scales Nominal scales Ordinal scales Interval scales Ratio scales

29 29 Nominal scales Labels and categorical data It involves numbering of different categories on which h no numerical operation or comparison could be applied. For example: Type of enterprises (TYPE) 1 = SOE 2 = JV 3 = WOFE For example: Do you think joining WTO will help the Chinese economy? 1=Yes 2 = No

30 30 Nominal scale No mathematical operation can be done. Use of dummy variables. In correlation/regression, the number of dummies is equal to the number of categories minus one. If we have three types of firms: SOE, JV and WOFE Firm performance = b 0 +b 1 SOE + b 2 JV

31 31 Dummy variables If we have three types of firms: SOE, JV and WOFE Firm performance = b 0 + b 1 SOE + b 2 JV Firm performance = a 0 + a 1 JV + a 2 WOFE A comparison of JV vs. non-jv effect on firm performance A comparison of JV vs. (SOE or WOFE) effect on firm performance

32 32 Ordinal scale An ordinal scale is one in which: 1. A set of objects or people is ordered from most to least with respect to an attribute, 2. There is no indication of how much, in an absolute sense, any of the objects possess the attribute, and 3. There is no indication of how far apart the objects are with respect to the attribute

33 33 Ordinal scale No average, nor correlation with ordinal data Rank order correlation. For example, rank ordering of candidates in an interview (RANK) Ranker 1 Ranker 2 Ranker 3 Average ranking Tom = 1 B = 1 C = 1 Tom = 2 B = 2 B = 2 C = 3 Tom = 3 Tom = 2 D = 3 D= 4 C = 4 D = 4

34 34 Rank order correlation t whenn>20n>20 2 (1 ) / ( n 2) 6 2 d i 6(194) ( 2 1) 10(99) nn IQ Hr of TV/wk rank x i rank y i d i d 2 i Source:

35 35 Interval scale An interval scale is one in which 1.The rank-ordering of objects is known with respect to an attribute, and, 2.It is known how far apart the objects are from one another with respect to the attribute, but 3.No information is available about the absolute magnitude of the attribute for any object.

36 36 Interval scale The candidate is confident Disagree Agree For example, ratings of candidates in an interview (RATE) Rater 1 Rater 2 Rater 3 Average A= & - x &

37 37 Ratio scale A ratio scale is one in which 1. The rank-order of persons with respect to an attribute is known, and, 2. The intervals between persons are known, and in addition, 3. The distance from a rational zero is known for at least one of the persons ,, x,

38 38 Psychological Measurement Clarify some terms Four types of scales Rl Reliability bl 信度 Validity 效度 Measurement scales Effect of reliability on validity How to define a construct

39 39 Reliability & Validity Construct t Commitment??? Error the same every time validity Idi Indicator I am proud of (item) working in this company. x x i = t i + + i Error different every time (random) reliability

40 40 Constant error and random error Measurement items observable Construct abstract

41 41 Reliability & Validity Construct t Commitment Error the same every time validity Indicator (item) I am proud of working in this company. x??? x i = t i + i reliability Error different every time (random) E i ~ N( 0, 2 )

42 42 Another view p(x) x i = t i + i E i ~ N( 0, 2 ) 2 8

43 43 The concept of Reliability Reliability of a measurement procedure (used to measure a construct) refers to its freedom from random errors of measurement. Compare the following three thermometers which measure the boiling point (in o C) of the same glass of pure water at 5 different occasions. Trial Thermometer A Thermometer B Thermometer C Which thermometer is more reliable? Why? How then would you define reliability?

44 44 Definition of reliability 108 o C = 100 o C + 5 o C + 3 o C Observed boiling point True boiling point Systematic error Random error 104 o C = 100 o C + 4 o C Observed score True score Random error x i = t i + i N Cov (t i, i )=0 i i i Classical measurement model

45 45 Measurement Error Turnover intention (t) t t i x 3 x 2 x 1 = I always think about leaving. x ik = t i + ik 0 ik i = 1,, N k=123 1, 2,

46 46 Reliability defined d Reliability is the degree to which a measure is free from random error. Observed score = True score + Random error x i = t i + i Reliability cannot be assessed by the absolute size of this error term. Why?

47 47 Reliability defined Observed score = True score + Random error Variation of x i = t i + i measurement error x = t + each time when we measure the same employees r xx 2 t 2 x Variation of turnover intention of different employees Definition Reliability = true scores variance observed scores variance

48 48 Reliability defined Definition true scores variance Reliability = observed scores variance Turnover intention (t) x 3 x 2 x 1 t t i 0 ik x ik = t i + ik i = 1,, N k=123 1, 2,

49 49 Reliability Calculation Subject True score Error Observed score Tom Peter Sam May Jody Ruth John Brian Variance Reliability = r xx = =.96

50 50 r xx and r tx r tx r r r r r tx tx tx tx 2 tx Cov ( t, x ) t x Cov(, t t ) 2 t 2 r xx r 2 tx t x x Cov(,) t t t 2 t t x t x r xx x x ik = t i + ik i = 1,, N k = 1, 2, 3

51 51 Reliability in diagram x i = t i + + i Construct (commitment) Measure (OCQ) Note: One can only talk about the reliability of a measure. A construct does not have reliability. Measurement Error (unreliability)

52 52 Measuring reliability According to this definition of reliability, r xx = r x1 x 2 r xx 2 t 2 x That is, reliability of a measure (r xx ) = correlation lti of ftwo parallel l measures of fthe same construct.

53 53 Measurement of reliability 1. Parallel forms reliability - using two test forms that are parallel to each other. Parallel forms are forms with same mean, variance, and same correlations with other measures. 2. Test-retest reliability -using the same test tform again on the same subjects at a different time. Reliability is the correlation between the scores of the same subjects on the two test runs. 3. Internal consistency reliability - splitting a test (of many items) into two equivalent halves so that they work as parallel forms.

54 54 Measurement of reliability Test-retest reliability agreement in different trials Parallel forms reliability agreement between left hand and right hand Internal consistency reliability agreement among the fingers

55 55 Test-retest reliability Subject Apr 2008 Jul 2008 Peter Sam May Jody Ruth John Brian Betty Reliability = = r xx = r 12 Mean Variance

56 56 Derivation Derivation t x t x x x ; t t ) x, Cov(x ; x x t t t t x t x t ) x Cov(x ), Cov( ), Cov(x ), Cov(x ) x, Cov(x ), Cov( t t t Var ) (x ) x, Cov(x t 2 x r xx Obs True r var var 2 1 t t x x 2 x x x x x x x

57 57 Parallel Measures Definition of parallel measures 1. Both are measuring the same construct 2. Both have the same true scores 3. Both have the same mean scores 4. Both have the same variance 5. N

58 58 Items from two parallel tests Form 1 (1) Which one of the following pair of words has opposite meaning? (a) regress-congress (b) frugal-luxurious (c) discreet-prudent (d) restless-faithful (e) cynical-judgmental (2) if y = 2x + 1 and x = 3y - 8, then (a) x=1 (b) x=2 (c) y=1 (d) y=2 (e) x=3 Form 2 (1) Which one of the following pair of words has opposite meaning? (a) digress-disgress (b) pensive-talkative (c) discreet-careless (d) restless-introspective (e) cynical-moody (2) if 2x + y = 7 and x = 2y - 4, then (a) x=1 (b) x=2 (c) y=1 (d) y=2 (e) x=3

59 59 Parallel Form Reliability Subject Peter Sam May Jody Ruth John Brian Betty Form Form Reliability = r xx = = < r xx < 1 r 12 Mean Variance

60 Splitting a test into 2 halves Supervisory rating for one candidate: 1. His work efficiency is high His work quality is good His work performance is good He always finishes his work assignments He always accomplishes his in-role tasks He always achieves his work objectives. 4 Sum A = 15 Sum B = 14 Note: 1=disagree; 7=agree

61 61 Split-half reliability Subject 1 st half 2 nd half Peter Sam May Jody Ruth John Brian Betty Mean Variance Reliability = r xx = = r 12

62 62 Correction for length of scale r kk 1 ( r k r xx is the reliability of the original scale; xx r kk is the reliability of the new scale; Length of the new scale is k-times that of the k 1) rxx original scale. If a 20-item scale measuring a construct has r xx of.70 We can add in 40 new items (measuring the same scale) and increase the r xx of the new 60-item scale to : 3 (.70) r kk (3 1)(.70)

63 63 Split-half reliability 1. His work efficiency is high His work quality is good His work performance is good He always finishes his work assignments He always accomplishes his in-role tasks He always achieves his work objectives. 4 standardized x ik = t i + ik Internal Consistency n! 6! 6*5*4*3*2*1 15 r!( n r)! 2!4! (2*1)(4*3*2*1) reliability 3 rij 1 2 r ij = α

64 64 Coefficient Cronbach Cronbach The average of all possible standardized split halve reliabilities. n t ( n 1 2 n i1 2 1 t 2 i ) i th item t = x i i 2 t 2 n is the number of items for each subject t is the total of all items for a subject is the variance of the ith item across all subjects is the variance of the total score (sum of all items) across all subjects

65 65 An example of coefficient Subject Item A B C Variance i11 Total Variance of total = 7.0 ; Total of variance = n 2 2 t n i i ( ) ( ) n 1 t Variance of item 2 2 i 2 t

66 66 The meaning of Cronbach n n 2 2 t i i 1 n i j 2 2 t n t ij n ( ) ( ) n1 1 Agreement among items in the scale

67 67 Effect of number of items Subject Item A B C Variance i Total Variance of total = 28.0 ; Total of variance = t n ( n 1 n i1 2 t 2 i ) ( ) i 2 t

68 68 Reliability Estimates Test-retest Time 1 Time 2 Form 1 Part 1 Internal Consistency Parallel form Part 2 Form 2

69 69 Reliability Values Cronbach s Alpha.80 ~ 1.0 Acceptable.70 ~.80 Marginal <.70 Questionable Test-retest ttreliability Could be as low as.50 ~.60

70 70 Psychological Measurement Clarify some terms Four types of scales Reliability 信度 Validity 效度 Measurement scales Effect of reliability on validity How to define a construct

71 71 Reliability & Validity Construct t Commitment Error the same every time validity??? x i = t i Indicator I am proud of working in this company. x reliability Error different every time (random)

72 72 Validity adydefined ed A measuring instrument is valid if it does what it is intended to do. Validity is the extent to which a measurement procedure actually measures what it is purported to measure. Construct Measures 1. How many flowers do you get on Love Valentine s Day. 2. How much sacrifice s/he is willing to make for you. Which one on the right has the highest validity? 3. How much s/he care about you. 4. How much s/he would share with you. 5. Whether s/he would die for you.

73 73 Validity Defined Validity involves expert judgment. Construct Measures 1. How many flowers do I get on Valentine s Love Day. 2. Whether s/he would die for you. 3. How much sacrifice s/he is willing to make for you. Which one on the right has the highest validity? 4. How much s/he care about you. 5. How much s/he would share with you.

74 74 Valid measure of performance Which one is the most valid measure of performance of employees working in a production line? Quantity of output Quality of output Number of rejects Cooperation with other colleagues Supervisor s subjective ratings Consistency of output Meeting supervisor s targets

75 75 Three views of validity Content validity Criterion validity Construct validity

76 76 Content validity

77 77 1. Content validity The content validity of a measure is demonstrated by showing that the measure constitutes a fair sample of the content of the construct to be measured. Perceptual speed quantitative verbal memory spatial quantity meeting targets quality variability Intelligence Job Performance

78 78 The content validity of OCB Altruism Willing to spend time to solve colleagues problems. Conscientiousness Willing to come back earlier and leave late. Sportsmanship Always criticize supervisor s s judgment and behaviors. Civic Virtue Provide constructive suggestions on improvements. Courtesy Proactively take actions to avoid conflict.

79 79 Content t Validity Content validity is a subjective judgment by Subject Matter Experts.

80 80 Criterion validity

81 81 2. Criterion validity Construct Happiness Longevity r=.56 Measure The average number of times one smile in a day

82 82 Criterion Validity Job performance r xy =.08 Is interview a valid measure of ability of the candidates? Scores on the interview (proxy of ability) (construct: ability; measure: interview score)

83 83 Criterion validity Job performance r xy =.08 In criterion validity, one uses a criterion variable (job performance) to determine the validity of a measure (interview score) of a construct (ability of candidates). Scores on the interview (proxy of ability) construct: ability measure: interview score

84 84 Criterion Validity The Logic: (1) The ability of a person should be highly correlated with his/her performance. (2) If we are measuring the ability of the candidates validly in the interview, the scores on the interview should be highly correlated with job performance. (3) Conversely, if scores on the interview are highly correlated with ihjob performance, we have evidence that we are validly measuring the ability of the candidates in the interview. Job performance r =.56 Scores on test A

85 85 criterion validity While criterion validity is a correlation coefficient, do not make up criterion validity with correlation. Criterion validity is just one possible use of the correlation coefficient to answer the question of the validity of a measure.

86 86 3. Construct validity I am capable of doing many things. Self Efficacy Discriminant validity Convergent validity Self Awareness Self Esteem I know what kind of person I am. I think I am a valuable person.

87 87 Construct validity Which constructs should I compare? How big (small) a correlation would be appropriate? y 1 Self Self Efficacy I am capable of doing y Awareness 2 many things. x Self Esteem Convergent validity I think I am a valuable person. r xy1 rxy 2 Discriminant validity I know what kind of person I am.

88 88 Construct validity The conceptual framework which specifies the meaning of the structure, distinguishes it from other constructs, and indicates how measures of the construct should relate to other variables is called construct validity. (1) Convergent validity two measures of the same construct should be highly correlated with each other. A new measure which is highly correlated with a known and valid measure of the same construct is said to have high convergent validity. (2) Discriminant validity two measures of different constructs should not be highly correlated with each other. A new measure of a construct should not be highly correlated with a known and valid measure of another construct. Such a measure is said to have high discriminant validity.

89 89 Convergent & Discriminant validity OCQ JDS (1) OCQ to measure organizational commitment (established) You want to show the validity of this new measure of job satisfaction (2) JDS to measure.55 Discriminant validity job satisfaction (established) (3) New measure of job satisfaction Convergent validity

90 90 Convergent & Discriminant validity OCQ JDS (1) OCQ to measure organizational commitment (established) You want to show the validity of this new measure of job satisfaction (2) JDS to measure job if i Discriminant satisfaction.55 validity (established)? (3) New measure of job satisfaction ( 中大工作满足感测验 ).58?.50 Convergent validity

91 91 Multi-trait-multi-method trait method matrix The multi-trait-multi-method (MTMM) matrix is simply a table displaying the correlations between: 1. The same traits measured by the same methods 2. The same traits measured by different methods 3. Different traits measured by the same methods 4. Different traits measured by different methods

92 92 An example of the MTMM matrix CEO Self report CEO Self report CEO Self report of firm innovation of market share R xx Firm innovation Market share.23 Discriminant Validity.92 Accounting report Convergent Firm innovation.73 Validity.15 Market share.12.62

93 93 MTMM for construct validity Self-rating Parent Rating Traits EI NEU EXT OPEN ANT CON EI NEU EXT OPEN ANT CON EI (.78) NEU -.39 (.77) EXT (.80) OPEN (.82) ANT (.83) CON (.86) Parent Rating EI (.81) NEU (.79) EXT (.83) OPEN (.85) ANT (.85) CON (.90) Heterotraitmonomethod Monotraitmonomethod Selfrating Heterotraitheteromethod Monotraitheteromethod

94 94 Interpreting MTMM Reliability (monotrait-monomethod) th should ldbe the highest Monotrait-heteromethod (convergent validity) must be >0 and high Monotrait-heteromethod > heterotraitmonomethod (discriminant validity) > heterotraitheteromethod

95 95 Psychological Measurement Clarify some terms Four types of scales Reliability 信度 Validity 效度 Measurement scales Effect of reliability on validity How to define a construct

96 96 Measurement Scale A scale measuring personal integrity 同意 没意见 不同意 只是不犯法, 歪曲法律並无不可 在学时我不只一次被校长处罚 大部分人会为了超越他人而说谎 偶尔一次说谎是人之常情 规则是为人而设立的, 被人打破也没什么大问题 把有价值的东西乱放的人, 应该与偷窃这些东西的人同受责备 Use of scales for reliability and validity concerns

97 97 Psychological Measurement Clarify some terms Four types of scales Reliability 信度 Validity 效度 Measurement scales Effect of reliability on validity How to define a construct

98 98 Effect of reliability on validity True test Observed True Observed Subject 王燕高庆华 score Score Performance 6 5 Performance 李鼎刘裕张三 赵梅陈庆新张兰 True Validity =.67 Observed Validity =.24

99 99 Validity Attenuation x x ; y y x t xy t x 1 t 2 r xx Cov( x, y ) (Definition of r ) 1 t 2 Cov(x,y ) Cov(x, ) Cov(y, ) Cov(, ) Cov(x,x) r xy t t xy t t t t t t 2 t t t x ty t xy rxy r r r r x y x y xx yy xx yy t t xx

100 100 Validity attenuation r xy t t r r xx xy r yy

101 101 Psychological Measurement Clarify some terms Four types of scales Reliability 信度 Validity 效度 Measurement scales Effect of reliability on validity How to define a construct

102 102 Multidimensional constructs Aggregate g Model Job Performance Quality Quantity On-time No. of rejects No. of outputs No. of delays Mental Ability Latent Model Math Verbal Memory What is x if x 2 +3x+5=0? +5 What is an ascetic? The sequence of a pile of random playing cards?

103 103 Convoluted constructs Psychological Contract (PC) Perceived Organizational Support (POS) Perceived Team Climate Perceived Insider Status (PIS) Perceived Supervisory Support (PSS)

104 104 How should we define a construct? 1. The "definitionalist" perspective essentially holds that the way to assure construct validity is to define the construct so precisely that you can operationalize it in a straightforward manner. In a definitionalist view, either you're measuring self esteem or you aren't. 2. The "relationalist" perspective holds that things are not black-and-white and -- concepts are more or less related to each other. The meaning of terms or constructs differs relatively, not absolutely. Your measure might be capturing a lot of the construct of self esteem, but it may not capture all of it. There may be another measure that is closer to the construct of self esteem than yours is. Relationalism suggests that meaning changes gradually. It rejects the idea that we can rely on operational definitions as the basis for construct definition. Trochim, William M. The Research Methods Knowledge Base, 2nd Edition. Internet WWW page, at URL: < (version current as of 10/20/2006 ).

105 105 Construct overlapping and nothing but self esteem Self worth Self confidence Self esteem Self disclosure Self efficacy Trochim, William M. The Research Methods Knowledge Base, 2nd Edition. Internet WWW page, at URL: < (version current as of 10/20/2006 ).

106 106 Operationalization In order to establish construct validity you have to meet the following conditions: You have to set the construct you want to operationalize (e.g., self esteem) within a semantic net (or "net of meaning"). This means that you have to tell us what your construct is more or less similar to in meaning. You need to be able to provide direct evidence that you control the operationalization of the construct -- that your operationalizations look like what they should theoretically look like. If you are trying to measure self esteem, you have to be able to explain why you operationalized the questions the way you did. If all of your questions are addition problems, how can you argue that your measure reflects self esteem and not adding ability? You have to provide evidence that your data support your theoretical view of the relations among constructs. If you believe that self esteem is closer in meaning to self worth than it is to anxiety, you should be able to show that t measures of self esteem are more highly correlated with measures of self worth than with ones of anxiety. Trochim, William M. The Research Methods Knowledge Base, 2nd Edition. Internet WWW page, at URL: < (version current as of 10/20/2006 ).

107 107 Discriminant/convergent validity F I am able to control my temper so that I can handle difficulties rationally. I am quite capable of controlling my own emotions. =.88 F I can always calm down quickly when I am very angry. I have good control of my own emotions.

108 Discriminant/convergent validity Look at the items 2. Inter-factor correlation 3. What is construct validity? =.88 F Convergent validity Discriminant i i validity F F2 d.f. = 1 ; = d.f. = 1 ; = 26.04** N = d.f. = 2 ; = 27.39

109 109 The End