Department of Chemistry Learning Assessment Plan: 2006-07 The Chemistry Department s Learning Outcomes Assessment Plan is included as Appendix 2. Based on how our concentrators have done in post-graduate schools (chemistry, biochemistry or health-care), the general, but informal and nonquantitative, sense is that our program is working. The small number of majors, and the variety of career paths they take, makes quantitative assessment of our graduates problematic. Our early efforts at assessment have focused on the two large-enrollment courses, the General Chemistry (101/102) and the Organic Chemistry (263/264) sequences. To date, our assessment of student learning has focused on student performance on final exams in the second courses in each sequence (Chem 102 and 264). Analysis of the student work in organic chemistry is based on an exam prepared by the Examinations Institute, an agency of the American Chemical Society, and given to students completing the organic chemistry sequence at universities around the country. Following the organic example, we used the ACS exam for Chem 102 in the spring of 2006. There were three fundamental problems with this assessment instrument (the ACS exam): Only a small number of universities participated in the ACS exam, so the ~ 150 Colgate students made up about 20% of the national sample An item analysis how did our students compare to the national average when answering questions about stoichiometry, or atomic structure, or acid-base chemistry, or electrochemistry, etc. was not available. The exam was profoundly flawed. It is not an exam we could give to our students with a clear conscience. With no meaningful national averages, and no item analysis available from their computers, we figured we might as well write our own exam, use it from year-to-year, and do the statistical work to determine which of the course goals were being met, and which weren t. We offered this new, home-grown exam for the first time this spring (2007). Chemistry Assessment page 1
Appendix 2 Chemistry Department s Learning Outcome Assessment Plan November 15, 2005 Dan Saracino, Director, Natural Sciences and Mathematics Roger Rowlett, Chair Department of Chemistry Learning Outcomes Assessment Plan The Department of Chemistry has formulated a plan for assessing learning outcomes as requested by the Dean of Faculty on October 26, 2005. The specifics of our assessment plan are outlined below. Some elements of this plan have already been implemented for other reasons, and limited historical data may already be available. It should be quite possible to begin implementation of all elements of the assessment plan as early as Spring 2006. Please note that the learning outcomes assessment plan outlined here is not intended to replace, but rather supplement our ongoing mission assessment utilizing perceptual surveys and senior exit interviews. Summary. The Department of Chemistry plans to assess learning outcomes at key points in the curriculum by regularly administering standardized examinations for the purpose of specifically examining student learning at the topical level in selected courses at all levels of instruction, from introductory (100-200 level) to advanced (300-400 level) courses. In addition, the Department will administer a questionnaire (see attached) and conduct structured exit interviews of all graduating seniors to obtain complementary information about courses not specifically subject to testing, and about the curriculum as a whole. Data obtained from assessment tools will be discussed annually by the entire faculty for the purpose of making appropriate pedagogical changes. Courses to be assessed. Assessment will be limited 1 to courses that have sufficiently large enrollment to ensure reasonable statistical averaging, and/or pertain to portions of the curriculum 1 We do not believe it is practical to perform a detailed assessment of every course in the department. To do so would require more time that can be reasonably spared from our teaching and research mission. However, because of the interdisciplinary nature of the chemistry curriculum, analysis of student performance in one course is likely to provide valuable information for revising other courses that examine related topics. Chemistry Assessment page 2
that are regarded as especially important by the American Chemical Society, the governing professional organization for chemistry and chemical education. All courses selected are large, introductory courses that serve a broad student constituency and/or courses that are specifically required for earning a bachelors degree in chemistry certified by the American Chemical Society. 2 Because we maintain an ACS-certified program in Chemistry, the latter courses are especially important to evaluate. A specific list follows: General Chemistry (CHEM 101-102) Students will be evaluated at the end of this one-year sequence in introductory chemistry that serves as a foundation course for the chemistry and biochemistry concentrations, and is a requirement for all students who wish to pursue postgraduate study in the health sciences. Organic Chemistry (CHEM 263-264) Students will be evaluated at the end of this one-year sequence in organic chemistry that serves as a foundation course for the chemistry and biochemistry concentrations, and is a requirement for all students who wish to pursue postgraduate study in the health sciences. Instrumental Methods (CHEM 371) Students will be evaluated at the end of this one-semester, laboratory-based course that is the gateway course for newly declared concentrators in chemistry and biochemistry. Proteins & Nucleic Acids (CHEM 353) Students will be evaluated at the end of this one-semester course that satisfies the American Chemical Society requirements that mandate exposure of all certified graduates to biochemistry. Advanced Inorganic Chemistry (CHEM 411) Students will be evaluated at the end of this onesemester course that satisfies the American Chemical Society requirements for exposure of all certified graduates to inorganic chemistry. Assessment instruments. Standardized examinations will be used to evaluate students in target courses. We believe this approach will be the most practical for several reasons. First, such examinations are relatively constant in format and topical coverage, which will allow for more meaningful longitudinal tracking. Second, the format of the examination allows for relatively simple examination and tabulation of student performance on specific questions or topical areas. Third, the examinations will be easy to administer. And finally, commercial examinations, where available and appropriate, are designed by professionals and can be normed against national standards of performance. Proposed assessment instruments are described below for each of the evaluated courses: General Chemistry (CHEM 101-102) An appropriate standardized examination for evaluating a full year of General Chemistry is available from the ACS. This examination can be administered during the regularly scheduled common examination period for CHEM 102 in the spring term. 2 Undergraduate Professional Education in Chemistry: Guidelines and Evaluation Procedures (2003) American Chemical Society, Washington, DC. Chemistry Assessment page 3
Organic Chemistry (CHEM 263-264) An appropriate standardized examination for evaluating a full year of Organic Chemistry is available from the ACS. This examination can be administered during the regularly scheduled common examination period for CHEM 264 in the spring term. This examination has been in regular use for CHEM 264 for many years, and historical data is thus already available for this course. Instrumental Methods (CHEM 371) An appropriate standardized examination for evaluating this course is available from the ACS. This examination can be administered during the regularly scheduled examination time for this course. This examination has been used periodically over the last decade, and some historical data is already available for this course. Proteins & Nucleic Acids (CHEM 353) An appropriate standardized examination for evaluating this course is not available from the ACS. However, an internally developed standardized examination has been administered periodically in this course for over a decade, and can provide the same kind of information available from commercially available standardized exams. This examination will be administered during the regularly scheduled time for this course. Some historical data is already available for this course. Advanced Inorganic Chemistry (CHEM 411) An appropriate standardized examination will be developed for this course that will allow for meaningful longitudinal tracking and item or topical analysis of performance. This examination will be administered at the regularly scheduled time for this course. Senior Questionnaire and Structured Exit Interviews. Questionnaires will be administered to all graduating seniors to assess their perceptions of the overall chemistry and biochemistry curricula, and to assess how graduating students personally value various components of the chemistry and biochemistry programs. This instrument has been in use for four years already. In addition we will administer structured exit interviews, in part to collect routine information from students about postgraduate plans, but mostly to talk confidentially about the strengths and weaknesses of the chemistry and biochemistry curricula that are not easily captured by a questionnaire. Analysis of assessment data. It is paramount to recognize that maximization of student performance on the standardized examinations described here is not the goal of chemistry courses or our curriculum in chemistry or biochemistry. Likewise, the administration of these examinations during the final exam period does not imply that these tools are measuring all that we wish our students to have accomplished during the year or term. That would be too much to ask of any assessment tool. Rather, we intend to utilize these examinations to (1) assess the basic factual and conceptual knowledge of our students in target courses, (2) compare the performance of our students against national norms established by our governing professional society, and (3) scrutinize the topical strengths and weaknesses of our students via item-analysis of student performance on exams. The first item gives a crude measure of the basic working knowledge of students enrolled in a particular course. The second item will keep us aware of the standing of our students relative to others nationwide. And most importantly, the third item will give us guidance as to the strengths and weaknesses in our pedagogical approach in target courses. Chemistry Assessment page 4
The results of our periodic 3 assessment will be made available to department faculty on an annual basis for discussion. Areas of suspected weakness as suggested by our assessment instruments can be considered and means of addressing these weaknesses can be devised for immediate implementation. 3 For most target courses, assessment will be conducted annually. However, we reserve the option of performing assessment evaluations on a less frequent basis. Chemistry Assessment page 5
Appendix 3 Assessment of Learning in Organic Chemistry Learning Goals Organic chemistry is the study of the properties, reactivity, preparation, and characterization of molecules comprised of carbon. Chem. 264 builds on and extends the fundamental principles developed in Chem. 263. In Organic Chemistry II, emphasis is placed on the following topics. 1. electronic structure and bonding (bonding theories, orbitals, hybridization, types of bonds, resonance, hyperconjugation, induction) 2. properties of molecules (acidity, basicity, bond strength, physical properties, stability, polarity, nucleophilicity, electrophilicity, leaving groups) 3. molecular structure (conformations, cycloalkanes, isomers, stereochemistry, strain, steric effects) 4. reactions (acid/base, substitution, elimination, addition, oxidation, and reduction as applied to all major functional groups) 5. mechanism (arrow pushing, free energy diagrams, thermodynamics, kinetics, transition state, intermediates, carbocations, carbanions, carbenes, radicals, Hammond postulate, polar reactions, radical reactions, experiments to establish mechanism, impact of reaction parameters) 6. synthesis (multi-step conversion of starting material to product, retrosynthesis) 7. spectroscopy ( 13 C NMR, 1 H NMR, EI-MS, IR, UV-vis) 8. communication (nomenclature, terminology, molecular depictions) 9. conjugation and aromaticity (properties of conjugated and aromatic compounds, aromaticity, heterocyclic aromatic compounds, reactions of aromatic compounds) 10. bioorganic chemistry (nucleic acids, lipids, carbohydrates, amino acids) Chemistry Assessment page 6
The aforementioned topics complete the foundation of organic chemistry. Proficiency in these topic areas is required for a student to successfully progress to further study of organic chemistry (Chem. 462) or to apply principles of organic chemistry to related fields of study (e.g., biochemistry, life sciences, health sciences). Assessment Instruments Four 50 minute unit examinations were administered during the semester to track student progress. Naturally, the exams were designed to assess proficiency in the key topic areas (learning goals) outlined above. It is quite common for students to obtain proficiency at different rates some students catch on quickly whereas others struggle initially but eventually hit their stride. As a result, the snapshots provided by the unit exams reflect each student s progress at a particular point in time, but may not reflect the student s eventual achievement. Thus, for purposes of overall assessment, a two-hour final examination was administered at the conclusion of the semester. The National Organic Chemistry Exam prepared by the Examinations Institute of the American Chemical Society Division of Chemical Education was used for this purpose. The exam is comprehensive to the full year of organic chemistry (Chem. 263/264). Use of the American Chemical Society National Exam allows year-to-year tracking of student performance as well as comparison of our student s proficiency to student performance nationwide. The exam was comprised of 70 multiple choice questions. The final examination counted towards 20% of each student s final letter grade. The assessment summary provided below pertains largely to student performance on the final examination. Chemistry Assessment page 7
Appendix 4 Assessment of Learning in General Chemistry Topics covered in Chem 102 Transition Metals Intermolecular Forces/ Solutions Chemical Kinetics Gaseous Equilibria Acids-Bases Buffers/K sp 2 nd Law of Thermo. Electrochemistry Nuclear Chemistry This was only the first year for this Final Exam. Chemistry Assessment page 8
Exit Questionnaire Department of Chemistry The following information is requested as part of ongoing assessment of the Department of Chemistry s programs in chemistry and biochemistry. Your participation with any part of this questionnaire is voluntary, and your responses will be kept confidential. Do not write your name on the form. The Department Chair will only release aggregate data. Please return the completed form to Lynda Case s mail slot in the department office. Biographical Information concentration(s) gender minor(s) ethnicity year of graduation Instructions In each of the following sections, circle the number that best reflects how strongly you agree with the following statements. (5 = strongly agree, 4 = agree, 3 = neither agree nor disagree, 2 = disagree, 1 = strongly disagree) Pre-Colgate 1. I was interested in science prior to coming to Colgate. 5 4 3 2 1 2. I was interested in (bio)chemistry prior to coming to Colgate. 5 4 3 2 1 3. I came to Colgate, in part, due to its known strength in the sciences. 5 4 3 2 1 4. I came to Colgate, in part, due to opportunities for student-faculty research. 5 4 3 2 1 5. I came to Colgate planning to concentrate in (bio)chemistry. 5 4 3 2 1 State your intended concentration upon enrollment at Colgate. Classroom and Laboratory Experience 1. The overall quality of teaching in the Department of Chemistry is high. 5 4 3 2 1 2. The overall quality of the laboratory experience (not including research) is high. 5 4 3 2 1 3. Lecture and laboratory were cohesive in chemistry classes. 5 4 3 2 1 4. My classroom experience generated enthusiasm for (bio)chemistry. 5 4 3 2 1 5. My laboratory experience (not including research) generated enthusiasm. 5 4 3 2 1 6. My laboratory experience provided appropriate preparation for research activities. 5 4 3 2 1 7. I was well prepared for later courses in (bio)chemistry by the earlier courses. 5 4 3 2 1 8. I feel well prepared for further study of (bio)chemistry if I were to choose to do so. 5 4 3 2 1 9. The overall (bio)chemistry curriculum is well-organized. 5 4 3 2 1 10. I received an appropriate level of attention and encouragement. 5 4 3 2 1 11. (Bio)Chemistry classes helped me to become a more independent learner. 5 4 3 2 1 12. (Bio)Chemistry classes improved my problem-solving skills. 5 4 3 2 1 13. (Bio)Chemistry classes improved my communication skills. 5 4 3 2 1 14. Department facilities (e.g. instrumentation) were appropriate to support my learning. 5 4 3 2 1 15. Integration of modern instrumentation enhanced my learning. 5 4 3 2 1
Senior Research (Chemistry 482) 1. I am happy with my Chemistry 482 research experience. 5 4 3 2 1 2. Chemistry 482 was an appropriate capstone experience in (bio)chemistry. 5 4 3 2 1 3. Chemistry 482 increased my knowledge of (bio)chemistry. 5 4 3 2 1 4. Chemistry 482 increased my confidence in my knowledge. 5 4 3 2 1 5. Chemistry 482 increased my critical analysis skills. 5 4 3 2 1 6. Chemistry 482 improved my problem solving skills. 5 4 3 2 1 7. Chemistry 482 improved my ability to read the chemical literature. 5 4 3 2 1 8. Chemistry 482 improved my ability to search the chemical literature. 5 4 3 2 1 9. Chemistry 482 improved my written communication skills. 5 4 3 2 1 10. Chemistry 482 improved my oral communication skills. 5 4 3 2 1 11. Chemistry 482 prepared me for future research should I choose to do so. 5 4 3 2 1 12. I feel that I was an intellectual partner in my research project. 5 4 3 2 1 13. I received adequate guidance from my faculty mentor. 5 4 3 2 1 14. My faculty mentor made his/her expectations clear to me. 5 4 3 2 1 15. My faculty mentor was encouraging and supportive. 5 4 3 2 1 16. I feel that my research project was original and significant. 5 4 3 2 1 17. I feel that my research project was challenging. 5 4 3 2 1 18. On average, I spent at least the required 8 hours per week working on my project. 5 4 3 2 1 19. My research abilities compare favorably to those of other students. 5 4 3 2 1 20. Department facilities were appropriate to support my research project(s). 5 4 3 2 1 Post-Baccalaureate Plans 1. I feel that I am well-prepared for my post-baccalaureate plans. 5 4 3 2 1 2. My post-baccalaureate plans were impacted by my (bio)chemistry courses. 5 4 3 2 1 3. My post-baccalaureate plans were impacted by my research experience(s) 5 4 3 2 1 4. I received adequate mentoring towards my post-baccalaureate plans. 5 4 3 2 1 5. The quality of advice from Chemistry faculty has been high. 5 4 3 2 1 6. Faculty are knowledgeable about a variety of educational and career options. 5 4 3 2 1 General Impressions 1. I am happy with my choice of a (bio)chemistry concentration. 5 4 3 2 1 2. I would recommend the (bio)chemistry program at Colgate to a friend or family 5 4 3 2 1 member interested in (bio)chemistry. 3. The Department of Chemistry cares about my learning. 5 4 3 2 1 4. The weekly department seminar series was valuable. 5 4 3 2 1 5. I am pleased with the sense of community in the Department of Chemistry. 5 4 3 2 1 6. The department sponsors adequate social activities for students and faculty. 5 4 3 2 1
Mission Statement The Colgate University Department of Chemistry aspires to be a national leader in the education of undergraduates in Chemistry and Biochemistry within a liberal arts environment. We are committed to maintaining a high-quality, scholarly teaching faculty dedicated to the preparation of students to be capable of future leadership within and outside of the sciences. Toward that end, we provide a program of undergraduate education in Chemistry and Biochemistry that is designed to produce breadth and depth of disciplinary knowledge and foster skills in critical analysis, problem-solving, and communication. We believe that a combination of formal classroom instruction, informal discussions, teaching laboratories, and research laboratory experiences within a collegial community is necessary to satisfy these aims. Original, significant student-faculty research is essential for achieving our goals for both our students and faculty. 1. I believe that the Department of Chemistry fulfills the above mission statement. 5 4 3 2 1 Please feel free to write additional comments below. You are also invited to discuss any comments raised by this questionnaire at your exit interview. Please return the completed form to Lynda Case s mail slot in the department office. Thank you for your participation.