Program Annual Assessment Report

Program Annual Assessment Report 2017-18 Programs: Chemistry B.S. and Chemistry B.A. Date: 25, September 2018 Outcome: Students will exhibit peer-level understanding of chemistry content Number of Artifacts Collected: 139 Number of Artifacts Reviewed: 139 Reviewers: John Helms, Brian McFarland, Andy Thomas internal American Chemical Society, Division of Chemical Education Examinations Institute external How were the artifacts reviewed selected: All collected artifacts were selected for review. Describe how the artifacts were used to assess student learning. (If a rubric was used please attach it to the report.): This year s programmatic assessment for the Chemistry B.S. and Chemistry B.A. degrees assessed our Programmatic Outcome #1, Students will exhibit peer-level understanding of chemistry content. To assess this outcome, we used nationally-normed exams created and scored by the American Chemical Society s Division of Chemical Education Examinations Institute. Exams were administered to students at 1. the end of CHEM 121 (General Chemistry I; Fall semester; 64 students) 2. the end of CHEM 122 (General Chemistry II, Spring semester; 43 students) 3. the end of CHEM 202 (Organic Chemistry II; Spring semester; 32 students) For perspective, CHEM 121 and 122 is a year-long General Chemistry sequence which is taken by all students who are Chemistry B.S. or B.A. majors, all Biology B.A. and B.S. majors, and a smattering of other students (for instance, some science teaching majors). The Organic Chemistry sequence (CHEM 201 and 202) is a year-long sequence that is taken by all Chemistry B.S. and B.A. majors, as well as all Biology B.S. majors. These specific exams administered cover a large number of chemistry content areas (see below) and, for our chemistry courses at Morningside, are used partially to determine a student s grade in each of these courses by serving as the final exam in each of the respective courses listed here. These exams are created by the American Chemical Society; each exam consists of 70 (CHEM 121 & CHEM 202) or 65 (CHEM 122) multiple-choice questions and must be completed in 110 minutes. The specific exams used in each course included: CHEM 121: First-Term General Chemistry Exam Form 2015; CHEM 122: Second-Term General Chemistry Exam Form 2014; CHEM 202: Organic Chemistry Form 1994 More specifically, the content areas tested for each exam are listed in Table 1. For more information on these exams, please visit the American Chemical Society (ACS) course

2 assessment materials at: http://uwm.edu/acs-exams/instructors/assessment-materials/exams/ Table 1. Student learning sub-content areas tested for each ACS exam administered during the 16-17 academic year ACS Exam/course: Content areas assessed* atomic structure, IUPAC nomenclature, gases, stoichiometry, periodic trends, CHEM 121 bonding theories, molecular structure, thermodynamics, quantum mechanics, aqueous chemistry CHEM 122 acids & bases, electrochemistry, equilibrium, intermolecular forces, kinetics, solubility, thermodynamics acidity of organics, fundamentals of organic chemistry, organometallics, CHEM 202 alkanes, dienes, stereochemistry, nucleophilic substitutions, epoxy chemistry, IUPAC-nomenclature, radicals, alkene/alkyne, carbonyls, aromatics, spectroscopy, organic synthesis *content areas listed in each course are generally regarded as the standard sequence of chemical subjects covered in each respective course, essentially prescribed by the American Chemical Society, and are highly standardized across curricula in the majority of chemistry course sequences in the U.S. and, to a lesser extent, globally. We use these three exams as broad indicators of student learning for each of the two-course sequences. Because the Organic Chemistry 201/202 sequence is a year-long sequence, we have used the final exam from 202 for assessment of both of those courses, which covers organic chemistry content from the entire year-long sequence. Conclusions of the review regarding student learning in the outcome with rationale supported by evidence from the artifacts: For all three courses (CHEM 121, 122, and 202), Morningside College student raw scores were somewhat lower than the national average - most pronounced from the General Chemistry I & II sequence (CHEM 121 & 122). The lower scores seen in General Chemistry relative to previous years may be due to a change in the assessment instrument - both CHEM 121 and CHEM 122 used a new revision of the ACS exam which saw changes to content covered. Additionally the student population in those introductory courses may have received less preparation in mathematics and science prior to taking the course than in previous years. In future, we would like to implement a mathematics & chemistry pretest which assesses chemistry knowledge and problem-solving skills at the beginning of CHEM 121 such that incoming skill levels can be determined to help our assessment of student learning. Despite these issues, we are quite pleased with the performance of our students, particularly considering our students educational goals and backgrounds compared to the national pool of students who take these same exams. The national averages in Figures 1-3 contain a great deal of data points (students) from primarily larger universities, most of which have American Chemical Society certified majors, which are better staffed (having a minimum of five full-time chemists) and better resourced than us, particularly when it comes to scientific equipment. These programs

3 typically test mostly hard-science majors, and have a much higher percentage of chemistry majors than our chemistry courses at Morningside College. As one example, the majority of our Chemistry majors are seeking the B.A. in Chemistry (not the B.S.), and the national comparison group typically would have a higher number and percentage of Chemistry B.S. students than Morningside (since we are not ACS certified, students who have a strong interest in chemistry will typically not attend Morningside). In addition, these exams would typically not be given to Biology B.A. students at larger universities, whereas Morningside chemistry courses have a significant number of Biology B.A. and All-Science Teaching majors: these students are typically not particularly strong in the hard sciences. We believe that the national norm data are skewed upwards, as most larger institutions have higher percentages of students with stronger science backgrounds than Morningside students (for instance, chemical engineering students, chemistry majors, physics majors, and Biology B.S. students). Essentially, our Chemistry program is a support program for our well-enrolled Biology program. Nonetheless, we use the ACS exam national averages as a benchmark for our students learning in these chemistry courses, due to its strong reputation and suitability for assessment. It is also important to note that we use these ACS exams as an important tool for our programmatic assessment, not just course assessment or assigning student grades. Since we graduate only a few Chemistry B.S. students each year, and only a handful of Chemistry B.A. students (who are primarily double-majors with the Biology B.S., and typically do not have a strong interest in chemistry), we are unable to effectively assess student knowledge of chemistry during their senior year. We have broken down the ACS exam results into disciplinary categories (Figures 1-3). For all courses, all exam questions in a category were used to calculate the average scores for that category.

4 Figure 1. CHEM 121 (General Chemistry I) Comparison of discipline category scores from Morningside College relative to national average statistics from ACS.

Figure 2. CHEM 122 (General Chemistry II) Comparison of discipline category scores from Morningside College relative to national average statistics from ACS. 5 Figure 3. CHEM 202 (Organic Chemistry II) Comparison of discipline category scores from Morningside College relative to national average statistics from ACS. Suggestions (if any) to improve student learning relative to this outcome. If no suggestions explain why not: Figures 1-3 provide considerable, important insights into student learning in our chemistry courses. The scores reported for General Chemistry I & II were all lower relative to the national ACS average. As noted above this could be due to changes in the assessment instrument and/or differences in incoming skill levels of the students. For Organic Chemistry II, our students scored higher than the national average in one or more disciplinary categories, and near the national mean for several categories, with a few disciplinary categories showing lower than the national mean We believe that one of the major strengths of using these ACS exams is that students have a strong motivation to do well on them, since their individual scores are used to calculate part of the student s overall grade in each of the respective courses. We do intend on continuing to use these exams as part of determining students grades, which also provides us with effective data

6 for assessment of student learning. Since our sample sizes were relatively large (N=32 to 64 students), we have reasonable confidence that the data are representative of the strengths and opportunities in our chemistry curricula. As such, we can easily identify those categories where additional resources or pedagogical focus is warranted. For this coming academic year, we will increase our teaching focus on the three lowest scoring categories for each course: 1. General Chemistry I: quantum mechanics aqueous chemistry nomenclature 2. General Chemistry II: intermolecular forces electrochemistry thermodynamics 3. Organic Chemistry 1 and 2: nomenclature alkenes/alkynes synthesis It is important to note that while we feel fairly confident that our assessment instrument represents an accurate measure of our students strengths and weaknesses, we may be somewhat limited in our effectiveness to overcome some of our identified shortcomings in student learning. For instance, for our Organic Chemistry sequence, we do not have some of the appropriate equipment for teaching our students interpretation of spectroscopic data that larger institutions have, making it much more difficult to overcome this shortcoming in student learning. Respectfully submitted on 9-25-2018 by A. Thomas