PROGRAM EVALUATION REPORT The following information is organized by CACREP-Accredited Program Specialty Areas for

Transcription

1 PROGRAM EVALUATION REPORT For the academic year, DePaul Counseling Program faculty conducted a program evaluation by reviewing available data and surveying stakeholders. The following report summarizes data gathered from state and national examinations, alumni survey responses regarding CACREP Standards and the DePaul Counseling Program, supervisor evaluations of internship students, and demographic data. The obtained data allows for an honest assessment of the program and an opportunity to improve our Specialty Areas in order to best prepare our students at the highest level of excellence in knowledge of theories, skills and the Counseling Profession. The following information is organized by CACREP-Accredited Program Specialty Areas for Clinical Mental Health The program requires the completion of 90 credit hours (quarter). This includes a 100- hour practicum and a 600-hour internship. 127 students are currently enrolled in this specialty area. The pass rate on the 2017 National Counselor Exam from this specialty area was 94%. Surveys were sent to alumni, supervisors, and employers for program evaluation. The program evaluations assessed the core standards within the program based on the CACREP standards. Furthermore, the program evaluation measured the counseling competencies of our graduates. The data from these surveys are used to improve and modify the program in our pursuit for excellence. o Overall, with few exceptions, alumni reported being prepared for the counseling profession. The alumni indicated that attention needs to be given to preparation in career development, assessment, human growth and development, and management and evaluation of counseling programs (see overview below). Core Standard Very unprepared Unprepared Prepared Very prepared 1. Core foundational counseling knowledge Current trends and changes in the counseling profession Ethical understanding and behavior

2 4. Multicultural competence Knowledge of how social system and policies affect underrepresented groups Advocacy Skills Counseling assessment and interventions Diagnosis and treatment planning Individual counseling Group counseling Knowledge of human growth and development Knowledge of career development Crisis Management Management and evaluation of counseling programs o Supervisors were asked to indicate to what degree specific competencies were evident among internship students (i.e., not evident, sometimes evident, consistently evident, and evident at an exemplary level). There were between 3-7 questions for each competency area. Overall, with few exceptions, supervisors indicated students were consistently exhibiting these competencies. Based on these results, additional attention could be focused on theory integration and case conceptualization. Below is an average rating for each of the sections. The full report can be found in Appendix A. 2

3 Sometimes evident Consistently evident Evident at an exemplary level Clinical Mental Health Sections Section 1: Therapeutic Relationship Competencies Section 2: Conceptual Competencies Section 3: Theory Integration Competencies Section 4: Supervisee Competency Section 5: General Case Management Competencies Section 6: Professional & Ethical Competencies NA 3

4 School Counseling The program requires the completion of 72 credit hours (quarter). The program intends to move to 90 credit hours in This includes a 100-hour practicum and a 600- hour internship. 103 students are currently enrolled in this specialty area. The pass rate for the State of Illinois School Counselor Licensure Exam was 100% Surveys were sent to alumni, supervisors, and employers for program evaluation. The program evaluations assessed the core standards within the program based on the CACREP standards. Furthermore, the program evaluation measured the counseling competencies of our graduates. The data from these surveys are used to improve and modify the program in our pursuit for excellence. o Overall, with few exceptions, alumni reported being prepared for the counseling profession. The alumni indicated that attention needs to be given to preparation in diagnosis and treatment planning and crisis management (see overview below). Core Standard Very unprepared Unprepared Prepared Very prepared 1. Core foundational counseling knowledge Current trends and changes in the counseling profession Ethical understanding and behavior Multicultural competence Knowledge of how social system and policies affect underrepresented groups Advocacy Skills Counseling assessment and interventions Diagnosis and treatment planning

5 9. Individual counseling Group counseling Knowledge of human growth and development Knowledge of career development Crisis Management Management and evaluation of counseling programs Awareness of datadriven comprehensive developmental schoolcounseling programs Strategies to promote and enhance the academic, career, and social/emotional development of K-12 students

6 o Supervisors were asked to indicate to what degree specific competencies were evident among internship students (i.e., not evident, emerging, proficient). There were between 3-18 questions for each competency area. Overall, supervisors indicated students were emerging or proficient at almost all competencies. Below is an average rating for each of the sections. The full report can be found in Appendix A. Competencies Not Evident Emerging Proficient NA Section 1: Professional Orientation Section 2: Foundation Section 3: Delivery Section 4: Management Section 5: Accountability

7 College Student Development The program requires the completion of 72 credit hours (quarter). The program intends to move to 90 credit hours in This includes a 100-hour practicum and a 600- hour internship. 48 students are currently enrolled in this specialty area. Surveys were sent to alumni, supervisors, and employers for program evaluation. The program evaluations assessed the core standards within the program based on the CACREP standards. Furthermore, the program evaluation measured the counseling competencies of our graduates. The data from these surveys are used to improve and modify the program in our pursuit for excellence. o Overall, alumni reported being prepared and very prepared for the counseling profession. The alumni indicate that attention needs to be given to preparation in diagnosis and treatment planning, knowledge of how social systems and policies affect underrepresented groups, career development, and management and evaluation of counseling programs (see overview below). Core Standard Very unprepared Unprepared 7 Prepared Very prepared 1. Core foundational counseling knowledge Current trends and changes in the counseling profession Ethical understanding and behavior Multicultural competence Knowledge of how social system and policies affect underrepresented groups Advocacy Skills Counseling assessment and interventions Diagnosis and treatment planning Individual counseling

8 10. Group counseling Knowledge of human growth and development Knowledge of career development Crisis Management Management and evaluation of counseling programs Awareness of datadriven comprehensive developmental schoolcounseling programs Strategies to promote and enhance the academic, career, and social/emotional development of K-12 students o Supervisors were asked to indicate to what degree specific competencies were evident among internship students (i.e., not evident, emerging, proficient). There were between 3-21 questions for each competency area. Overall, supervisors indicated students were emerging or proficient at almost all competencies. Below is an average rating for each of the sections. The full report can be found in Appendix A. Competencies Not Evident Emerging Proficient NA Section 1: Professional Orientation & Identity Section 2: Equity, Diversity and Inclusion Section 3: Delivery Section 4: Program Development and Evaluation Section 5: Legal and Ethical Knowledge

9 The following data includes all programs (Clinical Mental Health, College Student Development, and School Counseling. Job Placement Rate: The job placement rate for the graduating class of 2016 is 88%. Completion Rate The 6-year completion rate data for Master s in Counseling is 80.77% for the 2010 cohort. The 4-year completion rates for Master s in Counseling: 77.53% for the 2012 cohort. Number of graduates for past three years: Graduates Clinical Mental *16 (Community Health Counseling Counseling program) College Student Development School Counseling Total The following tables show the race and gender of students enrolled in the academic year. Race for Enrollment N American Indian 1 Asian African American 9 Hispanic Not Specified Pacific White Islander Total % Gender for Enrollment Sex Number Percent F M Total

10 Disposition Data The academic programs within the College of Education have set forth these dispositions as educational and professional expectations for all students. Dispositions are assessed a quarterly basis for all students in the Counseling Program. The highest number of dispositional concerns was found in the area of professional behavior. The lowest number of concerns was in the areas of consideration of cultural contexts, engagement in learning, initiative, and professional boundaries and receptiveness to feedback. Dispositions N* Concern Concern Concern Concern Not Observed Not Observed Observed Observed (N) (N) Appropriate Expression Consideration of Cultural Contexts Effective Communication Engagement in Learning Initiative Legal/Ethical Conduct Personal Responsibility Professional Behavior Professional Boundaries Receptiveness to Feedback Reflection * N = Total number of evaluations for (3 academic quarters) 10

11 Appendix A Completed Changes to Program In reviewing the program evaluation data, the DePaul Counseling Program faculty members have made a number of substantial changes to the program: 1. The leadership structure of the Counseling Program now includes an Academic Unit Leader, a Professional Practice Coordinator, and a CACREP liaison. 2. Students in the DePaul Counseling Program are assessed through Key Performance Indicators for each core curricular area and specialty area. Faculty members use the designated course-based assignments and practicum/internship rating forms to assess students using multiple methods and at multiple points in time. 3. The Clinical Mental Health Specialty Area increased the number of quarter hours from 72 to CLS 454 Career Counseling made changes to improve the student experience and to meet standards. The course now provides service learning, integrate theory and deliver workshops within the community, quizzes assess learning, flipped classroom and requires three taped counseling sessions. 5. The faculty made changes to courses which alumni indicated they were unprepared (i.e., assessment, human growth and development, management and evaluation of counseling programs, diagnosis and treatment planning, and crisis management). CSL 489 has standardized methodology for making diagnoses called the Interactive Practice Model. can create a workshop for supervisors, alumni for CEUs; The Crisis Intervention for School Counseling will be a 4 credit hours in the 90 credit hour curriculum to begin in the academic year. 6. The faculty surveyed alumni, site supervisors, and employers in order to gather additional information about the strengths and opportunities for growth within the Counseling Program. 7. In reviewing this survey data, a significant number of alumni across all three tracks identified Career Counseling as an area in which they were unprepared. The Counseling Program has hired a new faculty member who specializes in Career Counseling to teach this course in an effort to more adequately prepare our alumni to work with career related issues when they enter the field. 8. The faculty members have developed a plan for systemically evaluating the Counseling Program on a yearly basis. The faculty will meet in each Fall to examine various data points including supervisor, alumni, and employer evaluations, national and state exam data, supervisor evaluations of internship students, demographic and outcome data. The 11

12 program faculty will implement changes to improve the program based on analysis of the data. An annual report of the evaluation will be posted on the program website. 9. The Academic Unit Leader will quarterly review for accuracy the program materials on LiveBinder and the website. The Academic Unit Leader will report to the faculty any changes and/or updates to the LiveBinder and website at the 2 nd faculty meeting each quarter. Changes in Progress to the Program 1. The College Student Development and School Counseling Specialty Areas have plans in place to increase the number of quarter credit hours from 72 to 90 by The faculty have plans for reviewing the courses which alumni indicated they were unprepared (i.e., assessment, human growth and development, management and evaluation of counseling programs, diagnosis and treatment planning, and crisis management) to determine what curricular or staffing changes can be made. Two faculty have been charged with reviewing CSL 461, 489 and 484 which covers assessment, diagnosis, crisis and treatment planning. They will conduct a focus group with alumni to gather further information and how best to develop KPIs to improve student preparations. Faculty members have been assigned to examine the human development course and how best to include management and evaluation of counseling programs into the research and assessment courses. 12

13 Competencies Items N Mean Appendix B Supervisor Feedback on Core Competencies Community Counseling/Clinical Mental Health Not Evident Sometimes evident Consistently evident Evident at an exemplary level Section 1: Therapeutic Relationship Competencies Q Q Q Q Section 2: Conceptual Competencies Q Q Q Q Section 3: Theory Integration Competencies Q Q Q Section 4: Supervisee Competency Q Q Q Section 5: General Case Management Competencies Q Q Q Q Q Q NA 13

14 Q Section 6: Professional & Ethical Competencies Q Q Q Q Q Q School Counseling Competencies Items N Mean Not Evident Emerging Proficient Section 1: Professional Orientation Q Q Q Q Q Q Q Q Q Q Q Section 2: Foundation Q Q Q Q Section 3: Delivery Q Q Q Q NA 14

15 Q Q Q Q Q Q Q Q Q Q Q Q Q Q Section 4: Management Q Q Q Q Q Q Q Section 5: Accountability Q Q Q College Student Development Competencies Items N Mean Not Evident Emerging Proficient NA Section 1: Professional Orientation & Identity Q Q Q Q Q Q

16 Q Q Q Q Q Q Section 2: Equity, Diversity and Inclusion Q Q Q Q Q Q Q Q Q Q Q Section 3: Delivery Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q

17 Q Q Q Section 4: Program Development and Evaluation Q Q Q Q Q Q Q Q Q Section 5: Legal and Ethical Knowledge Q Q Q Q Q