TESTING AND MEASUREMENT IN PSYCHOLOGY Merve Denizci Nazlıgül, M.S.
PREPARING THE DATA FOR ANALYSIS 1. ACCURACY OF DATA FILE 1st step SPSS FREQUENCIES For continuous variables, are all the values within range? Are means and standard deviations plausible? If you have discrete variables (such as categories of religious affiliation), are there any out-of-range numbers? Have you accurately programmed your codes for missing values?
PREPARING THE DATA FOR ANALYSIS 2. MISSING DATA 1st step The pattern of missing data is more important than the amount missing. Missing values scattered randomly through a data matrix pose less serious problems. Nonrandomly missing values, on the other hand, are serious no matter how few of them there are because they affect the generalizability of results. Suppose that in a questionnaire with both attitudinal and demographic questions several respondents refuse to answer questions about income. It is likely that refusal to answer questions about income is related to attitude.
PREPARING THE DATA FOR ANALYSIS
PREPARING THE DATA FOR ANALYSIS
PREPARING THE DATA FOR ANALYSIS According to Little s MCAR test; he data were missing completely at random since the p-value is not significant (χ 2 = 3.64, df = 4, p =.46).
PREPARING THE DATA FOR ANALYSIS 2. MISSING DATA 2nd step Use Analyze > Descriptive Statistics > Frequencies Look at the frequency tables to see how much missing If the amount is more than 5%, there is too much. Need analyze further.
2. MISSING DATA 3rd step Replace with mean Mean replacement only for continuous variables PREPARING THE DATA FOR ANALYSIS
2. MISSING DATA 3rd step Replace with mean PREPARING THE DATA FOR ANALYSIS
PREPARING THE DATA FOR ANALYSIS
Scale for Measuring Attitudes Towards Mathematics or Science Strongly agree Agree Undecided Disagree Strongly disagree REVERSE SCORING 1. I want to develop my mathematical (science) skills and study this subject more. 2. Mathematics (science) is not a very interesting subject. 3. Mathematics (science) is a very worthwhile and necessary subject. 4. Mathematics (Science) makes me feel nervous and uncomfortable. 5. I have usually enjoyed studying mathematics (science) in school. 6. I don t want to take any more mathematics (science) than I absolutely have to. 5 4 3 2 1 5 4 3 2 1 5 4 3 2 1 5 4 3 2 1 5 4 3 2 1 5 4 3 2 1 7. Other subjects are more important to people than 5 4 3 2 1 mathematics (science). 8. I am very calm and unafraid when studying 5 4 3 2 1 mathematics (Science). 9. I have seldom liked studying mathematics (Science). 5 4 3 2 1 10. I am interested in acquiring further knowledge of mathematics (science). 5 4 3 2 1
REVERSE SCORING BUT FIRST; REVERSE SCORING E.g; item 2 (Mathematics (science) is not a very interesting subject.) was phrased the opposite way. Therefore, if you have reverse phrased items then you have to also reverse the way in which they re scored before you conduct reliability analysis. Positively worded items from the questionnaire: 1, 3, 5, 8, 10 Negatively worded items from the questionnaire: 2, 4, 6, 7, 9
REVERSE SCORING REVERSE SCORING 1 = strongly disagree 2 = disagree 3 = undecided 4 = agree 5 = strongly agree We need to reverse scale! 1 = strongly agree 2 = agree To reverse the scoring find the max value of your response scale (in this case 5) and add one to it. Then, for each person you take this value and substract from it the score they actually got. s/o who scored originally 5, now gets 6-1=5
TRANSFORM COMPUTE...
REVERSE SCORING- 2ND ALTERNATIVE
REVERSE SCORING- 2ND ALTERNATIVE
REVERSE SCORING-3RD ALTERNATIVE
REVERSE SCORING-3RD ALTERNATIVE
TOTAL SCORE OF A SCALE
TOTAL SCORE OF A SCALE
RELIABILITY ANALYSIS Cronbach's alpha ANALYZE SCALE RELIABILITY ANALYSIS
MODEL ALPHA RELIABILITY ANALYSIS Cronbach's alpha STATISTICS SCALE SCALE IF ITEM CORRELATIONS
RELIABILITY ANALYSIS Cronbach's alpha Cronbach Alpha Results: Alpha = 0.562 If deleting item 7, the Alpha will raise to 0.741. Corrected item-total correlation should not be <.3
SD = Strongly disagree, D = Disagree, N = Neither, A = Agree, SA = Strongly Agree S D D N A S A 1 Statistics makes me cry O O O O O 2 My friends will think I'm stupid for not being able to cope with SPSS. O O O O O 3 Standard deviations excite me. O O O O O 4 I dream that Pearson is attacking me with correlation coefficients. O O O O O 5 I don't understand statistics. O O O O O 6 I have little experience of computers. O O O O O 7 All computers hate me. O O O O O 8 I have never been good at mathematics. O O O O O 9 My friends are better at statistics than me. O O O O O 10 Computers are useful only for playing games O O O O O 11 I did badly at mathematics at school. O O O O O People try to tell you that SPSS makes statistics easier to understand 12 but it doesn't. O O O O O I worry that I will cause irreparable damage because of my incomptence 13 with computers. O O O O O Computers have minds of their own and deliberately go wrong 14 whenever I use them. O O O O O 15 Computers are out to get me. O O O O O 16 I weep openly at the mention of central tendency. O O O O O 17 I slip into a coma whenever I see an equation. O O O O O 18 SPSS always crashes when I try to use it. O O O O O 19 Everybody looks at me when I use SPSS. O O O O O 20 I can't sleep for thoughts of eigenvectors. O O O O O I wake up under my duvet thinking that I am trapped under a normal 21 distribution. O O O O O 22 My friends are better a SPSS than I am. O O O O O 23 If I am good at statistics people will think I am a nerd. O O O O O The SAQ
We will determine reliability for each of the 4 subscales separately: RELIABILITY ANALYSIS Cronbach's alpha Subscale of SAQ Items Fear of computers 6,7,10,13,14,15,18 Fear of statistics 1,3,4,5,12,14,10,21 Fear of mathematics 8,11,17 Fear of negative peer evaluation 2,9,19,22,23
Subscale 1: Fear of computers ANALYZE SCALE RELIABILITY ANALYSIS Transfer all items of 1 factor to the Item's window, here: Subscale 1 (Items 6,7,10,13,14,15,18). Proceed alike with the other 3 subscales
Reliability analysis on SPSS 'Scale if item deleted' tests whether alpha decreases if one item is deleted. In a reliable test, this should not matter much. We therefore expect still a high alpha (>.8) if an item is delete. Click OK
Subscale 1: 'fear of math' Item-total Statistics The individual values should not be greater than the overall since then their deletion would improve reliability! None of the items here affect reliability substantially The Corrected Item-total correlation give us the correlations between each item and the total score from the questionnare. In a reliable scale, all items should correlate with the total. Values should not be <.3. Average =.8234 (=Cronbach's ) Subscale 1 is reliable.
Subscale 4: Fear of negative peer evaluation Subscale 4, however, has a poor Cronbach's =.5699 Individual 's are not higher than the overall. Subscale 4 is not reliable. Q23 has low item-total correlation What's wrong with this subscale? It might have too heterogeneous questions which decreases internal consistency. This subscale should be rethought!
RELIABILITY ANALYSIS Split-half Reliability 2. split-half reliability: Split the scale in half and administer it to one subject Only need one administration. o The test items are divided into two halves, with the items of the two halves matched on content and difficulty. o If the scale is consistent, the subject should obtain 2 similar scores.
RELIABILITY ANALYSIS Split-half Reliability ANALYZE SCALE RELIABILITY ANALYSIS
MODEL SPLIT-HALF RELIABILITY ANALYSIS Split-half Reliability STATISTICS SCALE SCALE IF ITEM CORRELATIONS
RELIABILITY ANALYSIS Split-half Reliability Guttman Split-half = 0.6164 If deleting item it4, the Alpha will raise to 0.7860.