Validation of the Life Essentials Assessment Framework questionnaire

Transcription

1 Validatin f the Life Essentials Assessment Framewrk questinnaire Findings Gianfranc Giuntli Gary Raine Anne-Marie Bagnall April 2013 Centre fr Health Prmtin Research Institute fr Health and Wellbeing Queen Square Huse (Rm 230) 80 Wdhuse Lane Leeds Metrplitan University Leeds LS2 8NU g.giuntli@leedsmet.ac.uk 1 P a g e

2 Table f cntents Acknwledgments... 3 Executive summary Intrductin Backgrund Methds Analysis Discussin and recmmendatins...21 Appendix 1 - The Leaf questinnaire...24 Appendix 2 Descriptive statistics...35 Appendix 3 - Principal cmpnent analysis P a g e

3 Acknwledgments This study was funded thrugh a small grant frm AGE UK Wakefield District. It develped frm the invlvement f Prfessr Jane Suth in Wakefield Cmmissin n Pverty and Prsperity ( ). The authrs wuld like t thank her and the Centre fr Health Prmtin Research fr the supprt received. 3 P a g e

4 Executive summary Intrductin The Life Essentials Assessment Framewrk ( Leaf ) questinnaire is a six questins, interviewer-administered questinnaire devised by Age UK Wakefield District t enable effective evaluatin f vulnerable adults needs and t help establish the effectiveness f service prvisin. It aims t meet the increasing demand by funders t measure service users imprvement against expected delivery utcmes. Leaf cvers 6 areas f life, called paths, which are deemed essential t psitive living fr vulnerable adults: Path 1 - Daily Living Skills day t day living, self-care and persnal hygiene shpping Path 2 - Managing Finances: planning and managing finances, future plans Path 3 - Scial Netwrks: relatinships and scial cnnectins Path 4 - Emtinal Wellbeing: depressin, anger, dignity Path 5 - Physical Health: management f lng term cnditins, mbility etc. Path 6 - Pleasure in Life: satisfactin, interest in a variety f settings and circumstances. Each path is assessed thrugh a single questin which is exemplified by sme statements and is evaluated n a ten steps ladder. Leaf is administered at three pints in time: at the pint f assessment f the clients needs, at the cmpletin f the service delivered r at six weeks, depending n the nature f delivery, and then at 90 days. The Centre fr Health Prmtin Research, Institute fr Health and Wellbeing at Leeds Metrplitan University was cmmissined by AGE UK Wakefield District t assess the measurement characteristics f Leaf. The assessment aimed t investigate its validity, reliability, and capacity t measure change in tw phases: A first phase in which existing data is used t undertake all pssible relevant validatin analyses. A secnd phase in which further, specific data are cllected t undertake all the validatin analyses that are nt pssible based n the existing data. This dcuments reprts n the results f the first phase f the assessment, which was undertaken using an existing data set f the answers t the Leaf questinnaire f 99 lder peple wh were interviewed at tw pints in time: befre and after the delivery f sme specific scial care interventins. 4 P a g e

5 Methds The available data enabled us t undertake the fllwing tests f the validity, reliability, and capacity t measure change f the Leaf questinnaire: Fr validity (which indicates whether the Leaf questinnaire effectively measures what it is intended t measure): Face validity, which indicates whether, n the face f it, the instrument appears t be assessing the desired qualities (assessed thrugh a methdlgical and theretical analysis). Cntent validity, which cnsists f a judgment n whether the instrument samples all the relevant r imprtant cntent/dmains given its main aims (assessed thrugh a methdlgical and theretical analysis). Factrial validity, which assesses whether the factr structure f the questinnaire cnfrms t the theretical definitin f the cnstruct (assessed using Principal Cmpnent Analysis). Fr reliability (which indicates whether the Leaf questinnaire is able t measure cnsistently): Internal cnsistency, which describes the extent t which all the items in a given test measure the same cnstruct (assessed using Crnbach s alpha). Fr measures f change (which indicates the capacity f the Leaf questinnaire t detect change befre and after the interventins): Paired t-test fr nrmally distributed variables and Wilcxn Signed-Rank Test fr nn nrmally distributed variables. Findings Overall, the analyses shwed that: The six questins that make up the Leaf questinnaire present significant elements f ambiguity, bth in their wrding and in their answering ptins, and need t be amended. The six items f the Leaf questinnaire d nt represent a scale that measures a single cnstruct. Hwever, three items ( Pleasure in life, Emtinal wellbeing, and Scial netwrks ) shw the ptential t represent a shrt scale aimed at measuring the cnstruct mental well-being. The Leaf questinnaire recrded an imprvement f the clients satisfactin after the delivery f the interventins. Hwever, n inference can be made in relatin t whether such change was caused by the services delivered by AGE UK because the questinnaire was nt administered t a cntrl grup. 5 P a g e

6 In particular, the face validity analyses suggested that: All f the six items presented sme imprtant surces f ambiguity that represent a threat t the validity f the answers recrded using the questinnaire. In particular: The questins underneath each item (see Appendix 1), which aim t exemplify the six main questins, are a surce f majr ambiguity because they make unclear what questin the respndents are actually answering, whether the main item r ne f the exemplifying nes. The questin aimed at assessing physical health: Hw des yur physical health affect yur life and hw well d yu lk after yur wn health? is duble barrelled, that is it asks tw questins in ne. As such it is a surce f majr ambiguity fr the respndents. The exemplifying statements in the scales f 1 t 10 d nt always ffer an intuitive, clear and cnsistent interpretatin f the specific level f the scale t which they have been assigned. Fr example, the descriptrs fr the scres 3, 5, and 7 f the scale fr the scial netwrks item (item number 3) are respectively: Needing help, smetime lnely, Wanting mre cntact, and Have peple arund me, wuld smetimes like mre cntact, which have very similar meanings. On the ther hand, the labels fr the 10 pints ladder f the item n emtinal wellbeing (item number 4), make an incnsistent use f adverbs: the label f scre 5 is cntent at times, which uses a tempral adverb, whereas the label fr scre 7 is quite cntent, which uses a quantity adverb. The label cntent, which is the tp end f the 10 steps ladders used t recrd the clients state in relatin t each f the six main questins, may nt allw the mst effective use f the scales. The cntent validity analyses suggested that: The six items f the Leaf questinnaire seem t be designed t tap n three main underpinning cnstructs: Mental Well-Being, Functinal abilities, and Living standards. Hwever, f these three cnstructs, nly Mental well-being can be linked t three items, e.g. Pleasure in life, Emtinal wellbeing, and Scial netwrks, which are enugh t ptentially be a small scale that tap n bth the hednic cmpnent f mental well-being that is hw peple feel abut life (e.g. their emtins and satisfactin with life) and its eudaimnic cmpnent, that is hw peple functin in life, respectively frm a psychlgical and a scial pint f view. 6 P a g e

7 The tw ther dmains, i.e. Functinal abilities and Living standards, are assessed respectively thrugh tw items, i.e. Physical health and Daily living skills, and ne item, i.e. Managing finances, which are nt enugh t cnstitute measurement scales. The factrial validity analyses shwed that: One item ( Managing finances ), des nt crrelate with the thers and s wuld need t be drpped in rder fr the Leaf t becme a valid scale. Tw items, i.e. Daily Living Skills and Managing Finances, were significantly skewed in bth the befre and after administratins and made limited use f the 10 pints scale range, suggesting that there are issues with their wrding, with the labels given t their 10 pints scales, r bth. The Leaf questinnaire assesses three main sub dmains. This result was cnsistent with the hyptheses advanced thrugh the cntent analysis f the questinnaire. The three sub dmains identified thrugh the Principal Cmpnent Analysis can be called respectively: Mental well-being, which cnsisted f the items Pleasure in life, Emtinal wellbeing, and Scial netwrks. In particular, the item Pleasure in life measured the hednic cmpnent f mental well-being, whereas the items Emtinal well-being and Scial netwrks measured its eudaimnic cmpnent. Functinal abilities, which cnsisted f the item Daily living skills and Physical health. Living standard, which cnsisted f the item Managing finances. Althugh the first sub dmain culd represent a sub scale f the Leaf questinnaire, the ther tw, i.e. everyday functinality and physical health, cnsisted nly f ne item. Overall, single items tend t be less reliable than measurement scales t assess specific life dmains. The Crnbach s alpha f the subscale mental well-being was.725 fr the befre administratin and.860 fr the after administratin, suggesting that the internal cnsistency f this sub scale is nt cnsistently abve the recmmended threshld f P a g e

8 Cnclusins and recmmendatins The first phase f the validatin f the Leaf questinnaire aimed t undertake all pssible relevant validatin analyses f this measurement tl using the answers cllected frm 99 lder peple interviewed at tw pints in time: befre and after the delivery f specific AGE UK services. The analyses returned three main findings: The current versin f the Leaf questinnaire presents significant ambiguities in the way the questins are wrded and the 10 pints ladders are labelled. The six items f the Leaf questinnaire d nt represent a scale that measure ne single underpinning cnstruct. Hwever, they tap n three main cnstructs, which can be called Mental well-being, Functinal abilities, and Living standard. Only the three items that measure the cnstruct Mental well-being shwed the ptential t represent a shrt scale. On the ther hand, the cnstructs Functinal abilities and Physical health were measured respectively by tw items and ne item and s did nt represent scales. Overall, single items tend t be less reliable than measurement scales t assess specific life dmains. The Leaf questinnaire recrded an imprvement f the clients satisfactin after the delivery f the interventins. Hwever, n inference culd be made in relatin t whether such change was caused by the services delivered by AGE UK because the questinnaire was nt administered t a cntrl grup. Overall, this first phase f the evaluatin f the Leaf questinnaire has shwn the ptential f this questinnaire and it is recmmended t prceed with the secnd phase f the validatin. Key recmmendatins fr the secnd phase f the validatin f the Leaf questinnaire are: The six items need t be rewrded in such a way t remve all surces f ambiguity, which represent a threat t the validity f the questinnaire. With regard t this, it is suggested that the exemplifying questins listed underneath each f the six items are nt used t explain the main items t the clients. Their current use as explanatins fr the main items during the administratin f the questinnaire represents a majr surce f ambiguity with regard t what questins the clients are actually answering. Each f the six main questins f the Leaf questinnaire shuld be self-explaining, if further examples are needed t clarify them, then this means that the questins are still significantly vague/ambiguus. 8 P a g e

9 It is imprtant t decide what is the verall gal f the Leaf questinnaire. If its main aim is t measure change in relatin t a number f aspects f the life f AGE UK clients which are all deemed essential, despite the fact that they may be unrelated t each ther, e.g. Managing finances, then the findings that the six items f Leaf d nt represent a scale shuld nt be f cncern. Hwever, this des nt exclude that each f the three life dmains identified thrugh the Principal Cmpnent Analysis, i.e. Mental well-being, Functinal abilities, and Living standard, can be measured thrugh shrt, valid and reliable scales, especially cnsidering that single items tend t be less reliable than measurement scales t assess specific life dmains. With regard t this, as mentined, three items f the Leaf questinnaire already present the ptential t be a shrt scale fr the measurement f the cnstruct Mental well-being. It is suggested that tw shrt scales culd be created fr the tw ther cnstructs, i.e. Functinal abilities and Living standard, using the explanatry questins (currently listed underneath each item) as cmplementary items. Because each f these explanatry questins are strictly related t the items that they intend t exemplify, they may be used as questins tapping n thse same cnstructs. There is the need t undertake further tests f the reliability (e.g. measures f stability) and validity f the Leaf questinnaire. Fr example: Test-retest reliability, which entails the cmparisn f the Leaf questinnaires t ther, already validated questinnaires and scales that aim t measure similar cnstructs. Cnstruct validity, which entails the frmulatin f specific hypthesis t test whether Leaf allws researchers t make accurate inferences abut Age UK clients. The Leaf questinnaire shuld be administered t a cntrl grup t help establishing causal links between AGE UK interventins and the change that the Leaf questinnaire recrds between different pints in time. Finally, it is recmmended t adpt a simpler way t recrd the data cllected thrugh the questinnaire. Currently the data is recrded in an Excel spreadsheet using letters instead f numbers. It is suggested t recrd the scres f the clients n each questin using numbers, nt letters, which cannt be used t undertake any statistical calculatin. 9 P a g e

10 1. Intrductin This dcument presents findings frm the first phase f the assessment f the Life Essentials Assessment Framewrk ( Leaf ) questinnaire, a six questins, intervieweradministered questinnaire devised by Age UK Wakefield District t enable effective evaluatin f vulnerable adults needs and t help establish the effectiveness f service prvisin. The assessment aimed t investigate the validity, reliability, and capacity t measure change f the Leaf questinnaire in tw phases: A first phase in which existing data is used t undertake all pssible relevant validatin analyses. A secnd phase in which further, specific data are cllected t undertake all the validatin analyses that are nt pssible based n the existing data. This dcuments reprts n the results f the first phase f the assessment, which was undertaken using an existing data set f the answers t the Leaf questinnaire f 99 lder peple wh were interviewed at tw pints in time: befre and after sme specific scial care interventins. The remaining f this reprt is divided int fur parts. The first sectin ffers sme backgrund n the Leaf questinnaire. The secnd and third sectins reprt respectively n the methds and results f the first phase f the assessment, and the furth discusses the results and ffers sme recmmendatins fr pssible revisins f the Leaf questinnaire. 2. Backgrund The Leaf questinnaire s aims t meet the increasing demand by funders t measure service users imprvement against expected delivery utcmes. It cvers 6 areas f life, called paths, which are deemed essential t psitive living fr vulnerable adults: Path 1 - Daily Living Skills: day t day living, self-care and persnal hygiene shpping Path 2 - Managing Finances: planning and managing finances, future plans Path 3 - Scial Netwrks: relatinships and scial cnnectins Path 4 - Emtinal Wellbeing: depressin, anger, dignity Path 5 - Physical Health: management f lng term cnditins, mbility etc. Path 6 - Pleasure in Life: satisfactin, interest in a variety f settings and circumstances. Each life dmain (i.e. path) is assessed thrugh a single questin which is exemplified by sme statements and is evaluated n a ten steps ladder. The ladder cnsists f numbers 10 P a g e

11 frm 1 t 10, sme f which are assciated t example statements that change fr each life dmain (see Appendix 1). Leaf is administered at 3 pints in time: at the pint f assessment f the clients needs, at the cmpletin f the service delivered r at six weeks, depending n the nature f delivery, and then at 90 days. In each case, the service prvider and the client meet and discuss t find where the client fits n the scale. Leaf cllects als data n referral surces, reasns fr referral and agencies that individuals are signpsted n t. 3. Methds The data used fr the validatin cnsisted f the answers f 99 lder peple wh cmpleted the Leaf questinnaire n 2 ccasins: befre they were delivered specific scial care services and after their delivery. The available data enabled us t undertake the fllwing tests f the validity, reliability, and capacity t measure change f the Leaf questinnaire: Fr validity (which indicates whether the Leaf questinnaire effectively measures what it is intended t measure): Face validity, which indicates whether, n the face f it, the instrument appears t be assessing the desired qualities (assessed thrugh a methdlgical and theretical analysis). Cntent validity, which cnsists f a judgment n whether the instrument samples all the relevant r imprtant cntent/dmains given its main aims (assessed thrugh a methdlgical and theretical analysis). Factrial validity, which assesses whether the factr structure f the questinnaire cnfrms t the theretical definitin f the cnstruct (assessed using statistical techniques, please see belw). Fr reliability (which indicates whether the Leaf questinnaire is able t measure cnsistently): Internal cnsistency, which describes the extent t which all the items in a given test measure the same cnstruct (assessed using statistical techniques, please see belw). Fr measures f change (which indicates the capacity f the Leaf questinnaire t detect change befre and after relevant events): Paired t-test fr nrmally distributed variables and Wilcxn Signed-Rank Test fr nn nrmally distributed variables. 11 P a g e

12 In particular, fllwing the methdlgy suggested in the literature (see Field, 2005), the factr validity f the Leaf questinnaire was assessed by first checking the range, skewness, and standard deviatin f the six items. Skewness scres were standardized and z-scres with an abslute values greater than 2.58 were cnsidered t be significantly skewed at p <.001. Principal cmpnent analysis was then used. Given the general rule f having 10 participants per each variable (see Field, 2005), the sample f 99 participants can be cnsidered adequate t run Principal Cmpnent Analysis (the data set includes six variables, requiring a minimum sample f 60 participants). Hwever, verall, a sample f 100 participants is cnsidered belw ptimal levels (Field, 2005), s there is a pssibility that the analyses run int cmputatinal difficulties. A Varimax Rtatin was chsen n the basis f the fact that the three cnstructs identified thrugh the cntent analysis are theretically independent f each ther. Hwever, an blique rtatin was als used t check the hypthesis that the factrs were related t each ther. Given the sample s size and the fact that the analyses use less than 30 variables, Kaiser s recmmendatin f retaining factrs with eigenvalues ver 1 was adpted if cmmunalities after extractin were greater than 0.7 (Field, 2005). If nt, the scree plt was investigated and the number f factrs t retain decided based n the shape f the curve. Cnsidering the size f the sample (99 individuals), a lading greater than.512 was used t decide whether variables significantly laded n the factrs (Field, 2005). Crnbach s alpha was used as a measure f internal cnsistency. As cmmnly agreed in the literature, it was assumed that Crnbach s alpha shuld exceed 0.8. The available data did nt allw us t undertake further tests t ffer a mre in depth investigatin f the reliability (e.g. measures f stability such as test-retest reliability), validity (e.g. cnstruct validity), and capacity t measure change f the Leaf questinnaire (because there was n cntrl grup). 4. Analysis This sectin reprts n the analyses undertaken t explre the face validity, cntent validity, factr validity, reliability and capacity t measure change f the Leaf questinnaire. Face validity A review f the questins and respnse scales f the Leaf questinnaire indicates the fllwing issues: The questins in the bxes underneath each item (see Appendix 1), which aim t exemplify the six main questins, are a surce f majr ambiguity because they make unclear what questin the respndents are actually answering, whether the main item r ne f the exemplifying questins. The questin aimed at assessing physical health: Hw des yur physical health affect yur life and hw well d yu lk after yur wn health? is duble 12 P a g e

13 barrelled, that is it asks tw questins in ne. As such it is a surce f majr ambiguity fr the respndents. The exemplifying statements in the scales f 1 t 10 d nt always ffer an intuitive, clear and cnsistent interpretatin f the specific level f the scale t which they have been assigned. Fr example, the descriptrs fr the scres 3, 5, and 7 f the scale fr the scial netwrks item (item number 3) are respectively: Needing help, smetime lnely, Wanting mre cntact, and Have peple arund me, wuld smetimes like mre cntact, which have very similar meanings. On the ther hand, the labels fr the 10 pints ladder f the item n emtinal wellbeing (item number 4), make an incnsistent use f adverbs: the label f scre 5 is cntent at times, which uses a tempral adverb, whereas the label fr scre 7 is quite cntent, which uses a quantity adverb. The label chsen as the tp end f the 10 steps ladder, i.e. cntent, may nt allw the mst effective use f the 10 steps scale. Cntent validity The six items that cmpse the Leaf questinnaire tap n imprtant essential life dmains. Hwever, sme items, fr example Emtinal wellbeing, Pleasure in life, and Scial netwrks, can be intuitively related t the cnstruct f well-being and its tw main cmpnents, i.e. hednic well-being (which refers t hw peple feel abut life, e.g. their emtins and satisfactin with life) and eudaimnic well-being (which refers t hw peple functin in life psychlgically and scially). Other items, such as Managing finances, Daily living skills, and Physical health, aim t evaluate hw peple functin in different, mre practical aspects f their everyday life, s they can be cnsidered as wider indicatrs f quality f life. Overall, the items f the questinnaire seem t tap n the fllwing three main dmains: Mental well-being, which is brken dwn in its tw main cmpnents: Hednic well-being: item Emtinal wellbeing. Eudaimnic well-being: items Pleasure in life and Scial netwrks. Functinal abilities: items Physical health and Daily living skills. Living standard: Managing finances. In the Leaf questinnaire, nly the mental well-being dmain is assessed with three questins, whereas the ther tw are measured with ne r tw items at them mst, which are nt enugh t cnstitute measurement scales. Overall, single items tend t be less reliable than measurement scales t assess specific life dmains. Descriptive statistics and factrial validity The descriptive statistics f the six items f the Leaf questinnaire (see 0, Figure 2, and Table 1) shw that three items were significantly negatively skewed ( Daily Living Skills and Managing Finances bth in the befre and after administratins, whereas Physical Health nly in the after administratin) and ne, Emtinal Wellbeing, was significantly 13 P a g e

14 psitively skewed in the befre administratin. Figure 1 shws that the use f the lwer ends f the 10 pints scales was limited fr the items Daily Living Skills and Managing Finances. Overall, items that are significantly skewed, that make a limited use f the scale range, and that shw particularly high r lw values fr the standard deviatin indicate the presence f prblems and present an bstacle t undertake statistical analyses such as, fr example, factr analysis. Table 1. Skewness z-scres and standard deviatins Leaf items Daily Living Skills Managing Finances Scial Netwrks Emtinal Wellbeing Physical Health Pleasure in Life Befre z-scre After z-scre Befre Standard Deviatin After Standard Deviatin Inspectin f the crrelatin matrix (see Appendix 3) indicates that the item Managing Finances des nt crrelate with the majrity f the ther items. This fact suggests that this item is measuring smething different cmpared t the thers. Hwever, the determinant in the crrelatin matrix shws that there is nt a prblem with multicllinearity in the data (i.e. the items d nt crrelate very highly with each ther). The KMO statistic fr the Principal Cmpnent Analysis with all the variables in was.632 and the Barlett s test was highly significant, which indicates that factr analysis can be used fr this data, althugh the scre f the KMO statistic is cnsidered t be medicre (Field, 2005). Inspectin f the diagnal elements f the anti-image crrelatin matrix indicates that the item Managing finances has a value belw the requested 0.5, indicating issues with this item. These preliminary findings suggest that the item Managing finances shuld be remved frm the Leaf questinnaire if the aim is t build a valid scale. With regard t the number f factrs t retain, Kaiser s recmmendatin f retaining factrs with eigenvalues ver 1 wuld lead t retain tw factrs (see Table 2). Table 2 Cmpnent Initial Eigenvalues Ttal % f Variance Cumulative % P a g e

15 Hwever, despite the fact that the dataset included less than 30 variables, the cmmunalities after extractin were nt greater than 0.7 (see Appendix 3), s Kaiser s recmmendatin may nt be accurate fr this dataset. Inspectin f the scree plt (see Figure 1) suggests that a three factrs slutin might be mre apprpriate. S, bth a tw factrs mdel and a three factrs mdels were extracted. Figure 1. The tw factrs mdels explained a ttal f 64% f the variance, abut tw thirds f which was explained by the first factr (see Table 2). Factr 1, which can be called Mental well-being, cnsisted f the items Pleasure in life, Scial netwrks, and Emtinal well-being (see Table 3). Factr 2, which can be called Everyday functinality, cnsisted f the items Daily Living Skills, Physical health, and Managing Finances. Hwever, the tw factrs slutin presented a high prprtin (80%) f nnredundant residuals with abslute values higher than 0.5, which is a surce f cncern fr hw well the mdel fits the data. 15 P a g e

16 Table 3. Rtated Cmpnent Matrix a Items Cmpnent 1 2 PinL_Befre EW_Befre SN_Befre DLS_Befre MF_Befre PH_Befre Ntes. Extractin Methd: Principal Cmpnent Analysis. Rtatin Methd: Varimax with Kaiser Nrmalizatin. Rtatin cnverged in 3 iteratins. The three factrs slutin had als a KMO statistic f.632 and the Barlett s test was highly significant. The mdel explained 76% f the variance. The prprtin f nnredundant residuals decreesed t 60%, which, hwever, is still high and s a surce f cncern fr hw well the mdel fits the data. Table 4 shws that the three factrs were: Factr 1, Mental well-being, which cnsisted f the items Pleasure in life, Scial netwrks, and Emtinal well-being. Factr 2, Functinal abilities, which cnsisted f the items Physical health and Daily Living Skills. Factr 3, Living standard, which cnsisted f the item Managing Finance. The rthgnal rtatin and the blique rtatin ffered very similar results, s the Varimax methd was preferred and is reprted dwn here (see Table 4). Table 4. Rtated Cmpnent Matrix a Cmpnent PinL_Befre SN_Befre EW_Befre PH_Befre DLS_Befre MF_Befre Extractin Methd: Principal Cmpnent Analysis. Rtatin Methd: Varimax with Kaiser Nrmalizatin. a. Rtatin cnverged in 4 iteratins. 16 P a g e

17 Reliability Crnbach s alpha was calculated fr bth the Mental well-being (.725) and the Everyday functinality (.597) subscales identified by the first factr analysis (the Crnbach s alpha fr the questinnaires administered after the interventin was higher fr the cmpnent Mental well-being,.860, but nt fr the cmpnent every day functinality,.578). In bth cases it was suggested that remving sme f the items wuld have increased the verall alpha, indicating that nt all items psitively cntributed t the verall reliability. Hwever, even deleting the items did nt cause alpha t becme greater than 0.8, which is the prescribed ptimal value fr a scale t be reliable. Detecting change Because the six items vilated the assumptin f nrmality (see Figure 2), nnparametric Wilcxn signed-rank tests were used instead f paired t-tests t check whether AGE UK clients reprted higher satisfactin in the six life dmains after they were delivered relevant interventins. Table 5 shws that the clients reprted significantly higher satisfactin in the six life dmains after their received the relevant interventins. Hwever, Table 6 shws a high number f ties, that is f clients that did nt reprt any changes in their scres fr the six life dmains. Table 5. Wilcxn Signed Ranks Test DLS_After - DLS_Befre MF_After - MF_Befre SN_After - SN_Befre EM_After - EW_Befre PH_After - PH_Befre PinL_After - PinL_Befre Z b b b b b b Asymp. Sig. (2- tailed) b. Based n negative ranks. Table 6. Ranks DLS_After - DLS_Befre N Mean Rank Sum f Ranks Negative Ranks 0 a Psitive Ranks 30 b Ties 66 c Ttal 96 MF_After - MF_Befre Negative Ranks 1 d Psitive Ranks 40 e Ties 56 f Ttal 97 SN_After - SN_Befre Negative Ranks 0 g Psitive Ranks 57 h Ties 41 i 17 P a g e

18 N Ttal 98 Mean Rank Sum f Ranks EM_After - EW_Befre Negative Ranks 0 j Psitive Ranks 84 k Ties 15 l Ttal 99 PH_After - PH_Befre Negative Ranks 0 m Psitive Ranks 34 n Ties 59 Ttal 93 Negative Ranks 0 p PinL_After - Psitive Ranks 61 q PinL_Befre Ties 32 r Ttal 93 Ntes. a. DLS_After < DLS_Befre b. DLS_After > DLS_Befre c. DLS_After = DLS_Befre d. MF_After < MF_Befre e. MF_After > MF_Befre f. MF_After = MF_Befre g. SN_After < SN_Befre h. SN_After > SN_Befre i. SN_After = SN_Befre j. EM_After < EW_Befre k. EM_After > EW_Befre l. EM_After = EW_Befre m. PH_After < PH_Befre n. PH_After > PH_Befre. PH_After = PH_Befre p. PinL_After < PinL_Befre q. PinL_After > PinL_Befre r. PinL_After = PinL_Befre 18 P a g e

19 Figure 2. Histgrams f the six items f the Leaf questinnaire Daily Living Skills Managing Finances Scial Netwrks 19 P a g e

20 Emtinal Wellbeing Physical Health Pleasure in Life 20 P a g e

21 5. Discussin and recmmendatins The first phase f the validatin f the Leaf questinnaire aimed t undertake all pssible relevant validatin analyses f this measurement tl using the answers cllected frm 99 lder peple interviewed at tw pints in time: befre and after the delivery f specific AGE UK services. The analyses returned three main findings: The current versin f the Leaf questinnaire presents significant ambiguities in the way the questins are wrded and the 10 pints ladders are labelled. The six items f the Leaf questinnaire d nt represent a scale that measure ne single underpinning cnstruct. Hwever, they tap n three main cnstructs, which can be called Mental well-being, Functinal abilities, and Living standard. Only the three items that measure the cnstruct Mental well-being shwed the ptential t represent a shrt scale. On the ther hand, the cnstructs Functinal abilities and Physical health were measured respectively by tw items and ne item and s did nt represent scales. Overall, single items tend t be less reliable than measurement scales t assess specific life dmains. The Leaf questinnaire recrded an imprvement f the clients satisfactin after the delivery f the interventins. Hwever, n inference culd be made in relatin t whether such change was caused by the services delivered by AGE UK because the questinnaire was nt administered t a cntrl grup. Overall, this first phase f the evaluatin f the Leaf questinnaire has shwn the ptential f this questinnaire and it is recmmended t prceed with the secnd phase f the validatin. Key recmmendatins fr the secnd phase f the validatin f the Leaf questinnaire are: The six items need t be rewrded in such a way t remve all surces f ambiguity, which represent a threat t the validity f the questinnaire. With regard t this, it is suggested that the exemplifying questins listed underneath each f the six items are nt used t explain the main items t the clients. Their current use as explanatins fr the main items during the administratin f the questinnaire represents a majr surce f ambiguity with regard t what questins the clients are actually answering. Each f the six main questins f the Leaf questinnaire shuld be self-explaining, if further examples are needed t clarify them, then this means that the questins are still significantly vague/ambiguus. 21 P a g e

22 It is imprtant t decide what is the verall gal f the Leaf questinnaire. If its main aim is t measure change in relatin t a number f aspects f the life f AGE UK clients which are all deemed essential, despite the fact that they may be unrelated t each ther, e.g. Managing finances, then the findings that the six items f Leaf d nt represent a scale shuld nt be f cncern. Hwever, this des nt exclude that each f the three life dmains identified thrugh the Principal Cmpnent Analysis, i.e. Mental well-being, Functinal abilities, and Living standard, can be measured thrugh shrt, valid and reliable scales, especially cnsidering that single items tend t be less reliable than measurement scales t assess specific life dmains. With regard t this, as mentined, three items f the Leaf questinnaire already present the ptential t be a shrt scale fr the measurement f the cnstruct Mental well-being. It is suggested that tw shrt scales culd be created fr the tw ther cnstructs, i.e. Functinal abilities and Living standard, using the explanatry questins (currently listed underneath each item) as cmplementary items. Because each f these explanatry questins are strictly related t the items that they intend t exemplify, they may be used as questins tapping n thse same cnstructs. There is the need t undertake further tests f the reliability (e.g. measures f stability) and validity f the Leaf questinnaire. Fr example: Test-retest reliability, which entails the cmparisn f the Leaf questinnaires t ther, already validated questinnaires and scales that aim t measure similar cnstructs. Cnstruct validity, which entails the frmulatin f specific hypthesis t test whether Leaf allws researchers t make accurate inferences abut Age UK clients. The Leaf questinnaire shuld be administered t a cntrl grup t help establishing causal links between AGE UK interventins and the change that the Leaf questinnaire recrds between different pints in time. Finally, it is recmmended t adpt a simpler way t recrd the data cllected thrugh the questinnaire. Currently the data is recrded in an Excel spreadsheet using letters instead f numbers. It is suggested t recrd the scres f the clients n each questin using numbers, nt letters, which cannt be used t undertake any statistical calculatin. 22 P a g e

23 References Field, A. (2005) Discvering statistics using SPSS, Lndn: Sage. 23 P a g e

24 Appendix 1 - The Leaf questinnaire 24 P a g e

25 25 P a g e

26 26 P a g e

27 27 P a g e

28 28 P a g e

29 29 P a g e

30 30 P a g e

31 31 P a g e

32 32 P a g e

33 33 P a g e

34 34 P a g e

35 Appendix 2 Descriptive statistics DLS_Befre MF_Befre SN_Befre EW_Befre PH_Befre PinL_Befre N Valid Missing Mean Std. Errr f Mean Median Mde Std. Deviatin Variance Skewness Std. Errr f Skewness Kurtsis Std. Errr f Kurtsis Range Minimum Maximum DLS_After MF_After SN_After EW_After PH_After PinL_After N Valid Missing Mean Std. Errr f Mean Median Mde Std. Deviatin Variance Skewness Std. Errr f Skewness Kurtsis Std. Errr f Kurtsis Range Minimum Maximum P a g e

36 Appendix 3 - Principal cmpnent analysis Crrelatin Matrix a DLS_Befre MF_Befre SN_Befre EW_Befre PH_Befre PinL_Befre DLS_Befre MF_Befre Crrelatin SN_Befre EW_Befre PH_Befre PinL_Befre DLS_Befre MF_Befre Sig. (1-tailed) SN_Befre EW_Befre a. Determinant =.219 PH_Befre PinL_Befre KMO and Bartlett's Test Kaiser-Meyer-Olkin Measure f Sampling Adequacy..632 Apprx. Chi-Square Bartlett's Test f Sphericity df 15 Sig..000 Anti-image Matrices DLS_Befre MF_Befre SN_Befre EW_Befre PH_Befre PinL_Befre DLS_Befre MF_Befre Anti-image Cvariance SN_Befre EW_Befre PH_Befre PinL_Befre DLS_Befre.601 a MF_Befre a Anti-image Crrelatin SN_Befre a EW_Befre a a. Measures f Sampling Adequacy(MSA) PH_Befre a PinL_Befre a 36 P a g e

37 Cmmunalities Initial Extractin DLS_Befre MF_Befre SN_Befre EW_Befre PH_Befre PinL_Befre Extractin Methd: Principal Cmpnent Analysis. Ttal Variance Explained Cmpne Initial Eigenvalues Extractin Sums f Squared Rtatin Sums f Squared Ladings nt Ladings Ttal % f Cumulative Ttal % f Cumulative Ttal % f Cumulative Variance % Variance % Variance % Extractin Methd: Principal Cmpnent Analysis. 37 P a g e

38 Reprduced Crrelatins DLS_Befre MF_Befre SN_Befre EW_Befre PH_Befre PinL_Befre DLS_Befre.759 a MF_Befre a Reprduced Crrelatin SN_Befre a EW_Befre a PH_Befre a.483 PinL_Befre a DLS_Befre MF_Befre Residual b SN_Befre EW_Befre PH_Befre PinL_Befre Extractin Methd: Principal Cmpnent Analysis. a. Reprduced cmmunalities b. Residuals are cmputed between bserved and reprduced crrelatins. There are 12 (80.0%) nnredundant residuals with abslute values greater than P a g e