Zoom Out-Zoom In: Outcomes and Measures
Outcomes are about having a clear picture of what a graduate from your program should look like. What body of knowledge should they master, what skill set should they develop, what attitudinal dispositions should they possess.
ZOOM OUT for Outcomes
Writing learning outcomes for an academic program can be challenging. How do you sum up all that students should know in a handful of short statements? There are probably hundreds of “instructional objectives” that have been listed on course syllabi that lead to a degree in your degree program. The best way to think of outcome areas is to ZOOM OUT and see the big picture. What are the broad categories of knowledge, skills, and attitudinal dispositions that your program covers that will make them successful in their future endeavors? It can be helpful to think of a title for an outcome area, such as communication or critical thinking.
ZOOM IN for Measures
The goal of good assessment is to get a picture of students learning at the end of the program. This is why many programs have a capstone course that seeks to draw connections between learning from various courses, give a comprehensive review, and assess all that the student has learned and retained. Once you are able to see what students are really retaining, the work of assessment is to ZOOM IN on areas where student achievement is lower, so you can focus on ways you can shape the courses and experiences in the program to improve those areas. For a measure to have ZOOM IN capabilities, it must be able to be broken down into component areas.
Measures and ZOOM IN Capabilities
- Course Grades do not ZOOM IN and are not normally used in assessment.
- Test Grades, such as final exams, are not good measures of assessment by themselves. If questions can be grouped according to learning outcomes areas (components), then they can reveal patterns of overall strength and weakness in a cohort, which is the goal of assessment.
- Item Analysis of test questions helps detect patterns of response on individual questions, but the primary goal is usually to improve the quality of the questions themselves. Item analysis can be useful if they can be grouped around outcome areas.
- Question Group Analysis is something between individual item analysis and the test grade analysis for the class. This is the best ZOOM IN approach for assessing tests.
- Assignment Grades- Overall assignment grades demonstrate overall achievement of a cohort of students, but don’t help report the patterns of strength and weakness in the writing
- Assignment Rubrics- Rubrics can help ZOOM IN to component areas of the written work and define levels of proficiency. This can help the feedback given to students seem less subjective, and create a way for faculty to establish inter-rater reliability of assessments.