L&S Assessment: Tips and Terminology
Because many faculty and staff in L&S are not experts in higher education research, we provide this document in an effort to unpack the jargon and terms of art used in the field of "assessment". Much of the discussion below can (and likely will) be debated and disputed; please remember this is an introduction to the topic for people who are not (and who likely do not intend to become) expert in this field.
Because all UW-Madison academic programs are expected to be actively engaged in assessing student learning for purposes of program improvement, this document offers succinct definitions and explanations for some of the terms used in that endeavor. For questions about these processes as they relate to programs in the College of Letters & Science, please contact Associate Dean for Academic Planning, Elaine Klein (elaine.klein(AT)wisc.edu).
A (Limited) Glossary of Assessment Terms
NC State's list of Internet Resources for Higher Education includes several institutions' attempts to define various terms related to the assessment of student learning. (Search this page for "Glossary" to find nearly 40 examples; the list provided by Northern Illinois University is particularly comprehensive.) The list below focuses on terms used in the L&S Administrative Gateway and which are frequently used across campus. Please contact Elaine Klein through the comment box on this page to offer suggestions for improvement to this list, including terms that should be included.
Assessment / Outcomes Assessment / Assessment of Student Learning / Assessment for Improvement (of Learning)
These terms are generally regarded as interchangeable. Many leaders in higher education assessment have offered useful definitions for these terms.
"Assessment is the systematic
collection, review, and use of information about educational programs
undertaken for the purpose of improving learning and development."
(Palomba, C.A. & Banta, T.W. Assessment Essentials: Planning, Implementing, and Improving Assessment in Higher Education. San Francisco: Jossey-Bass, 1999, p. 4)
"Assessment is an ongoing process aimed at understanding and improving
student learning. It involves making our expectations explicit and
public; setting appropriate criteria and high standards for learning
quality; systematically gathering, analyzing, and interpreting evidence
to determine how well performance matches those expectations and
standards; and using the resulting information to document, explain, and
improve performance. When it is embedded effectively within larger
institutional systems, assessment can help us focus our collective
attention, examine our assumptions, and create a shared academic culture
dedicated to assuring and improving the quality of higher education."
(Thomas Angelo, AAHE Bulletin, November 1995, p. 7)
In a nutshell, the process requires that faculty and staff articulate or make explicit "learning outcomes" for academic or other programs; that they systematically gather information to better understand whether (and how well) students achieve those outcomes; and that they use that information to improve future results.
In assessing student learning in the academic program, the measure is taken of the academic program as a whole. This usually requires a different perspective, taking a broad view of the performance of groups of students enrolled in a whole course of study (e.g., a certificate program, a major, a degree program). Program-level assessment is different from assigning grades (i.e., an individual instructor's evaluation of an individual student); it is different from course evaluation (i.e., individual students' evaluation of the course in the context of the course); and it is different from teaching evaluation (i.e. faculty peers' evaluation of an individuals teaching skills).
Assessment for Accountability
At times, people will talk about "Assessment for accountability".
When assessment is used in this way, it is conducted to satisfy
stakeholders external to the unit. For example, some states and
professional certification boards may require training programs to
maintain a specific pass rate on an external certification examination
to maintain their approval. Results tend to be summative rather than
formative, and may be used to compare performance across similar units
to develop program rankings.
Assessment Plan / Assessment Report
All academic programs should
have assessment plans on file, and should be reporting assessment
results formally (via a free-standing report) or in the context of other
requests (e.g., when requesting program changes). The Office of the Provost oversees the Student Learning Assessment
Curriculum Map / Curriculum Mapping
An essential step in the assessment process, after having articulated learning outcomes for the program, is to identify the places in the curriculum where students will have opportunities to obtain that knowledge or learn those skills. Every course need not convey every outcome - but every outcome should be associated with a learning opportunity, and every student should have access to enough opportunities to ensure that they can achieve the program outcomes. Program-wide learning outcomes may be reinforced by noting in the course syllabus how particular projects or efforts support program learning outcomes (often, in a developmental way, by introducing, building on, and finally testing mastery of, those outcomes).
This strategy uses student work completed for a course for the purpose of program level assessment. For example, if all students in a program are required to take a particular course, they may be asked to respond to a common prompt related to program-level learning outcomes on an examination (across multiple sections and course offerings); the responses to that prompt could be examined periodically to discern whether or not students are responding as they should at that level. Or, capstone projects or Senior theses could be evaluated in light of program level learning outcomes.
Evaluation vs. Assessment
These terms are sometimes used interchangeably...and just as frequently, fine distinctions are drawn in multiple dimension that articulate clear differences between formative, process-oriented, reflective, and flexible Assessment, and summative, product-oriented, prescriptive, judgmental, and inflexible Evaluation. Those who adhere to these distinctions might assert that Assessment is about improvement over time, affecting groups of students on a grand scale; whereas Evaluation is about making a specific judgement at a particular point in time about an individual program or person. This distinction may be useful, so it seems best to ensure that these terms are defined when they're used.
Gathering information during a course or program of study, and using what is learned to improve results for those students. This is used frequently in classroom assessment, but can also be appropriate at the program level, particularly when the faculty want to gauge the impact of program changes.
When an instructor evaluates student work, that evaluation often takes the form of assigning a "grade" to that work. Grades usually aggregate many different elements that are evaluated, many of which are likely related to course or program learning outcomes that are intended to be revealed through the assignment. (And these may be evaluated using a tool such as a rubric, discussed below.) But other factors may also affect the assignment of a grade - timely completion, dealing with the assignment appropriately, improvement from draft to final versions, etc. But if all of these matters are managed carefully, grades may provide data that can be used for assessment. See L&S Assessment FAQ: Using grades as part of assessment strategy for more on this topic.
Learning Objectives, Goals, and Outcomes
These terms refer to what students should know or be able to do
upon completion of an academic program; all will vary according
to the program of study. A reasonable distinction can be made between these terms:
- "Objectives" is the term that is most frequently used to describe broad aspirations the faculty have for the program. For example, an objective may be that students who complete a particular program of study will be actively engaged as professionals in a particular field.
- "Goals" are somewhat more focused, and might describe - from the program's perspective - the general purpose of the program. It would be a "goal" for a program to focus on training prospective teachers, or to provide accredited training for a professional programs.
- "Outcomes" tend to be much more specific, and are usually framed from the perspective of the student. They usually include simple statements that describe as clearly as possible what students know or can do. For example, "when students complete
our program, they will be able to..." For purposes of assessment, it's
best that these outcomes reflect higher order thinking or skills that
are measurable; we strongly recommend that vague or aspirational
statements be avoided (if only because it's hard to measure something that is very aspirational,
like "our students change the world").
Learning Outcomes can be expressed at the program, course, and project level, and may therefore reflect different degrees of detail. It is important to remember that outcomes may only ever be a proxy for more sophisticated learning - that is, activities or knowledge that, if attained, reflect myriad other processes and abilities. (Every outcome need not be articulated in excruciating detail for the faculty to be assured that students have learned deeply and well.)
Measures (Direct / Indirect)
The means by which students' attainment of the learning outcomes/goals are measured. These are usually divided into two categories: "direct" and "indirect" measures.
- Direct measures look directly at students' knowledge and skills. These measures include - but are not limited to - test results, written assignments, presentations, classroom assignments, portfolio evaluation, performances, etc., that result from what they've learned in the class or program being evaluated.
- Indirect measures look at what students know or can do less directly. Indirect measures include such activities as surveys of students or their employers, job placement rates, subsequent entry into graduate programs, etc.
In general, assessment plans should include at least one (and preferably more) direct measure.
These tools are frequently used to evaluate student learning in complex categories of learning, usually in cases where achievement of a particular outcome may include several skills. (For example, rubrics are commonly used in evaluating student writing, where the mechanics of grammar and punctuation are evaluated seperately from the ability to structure an argument, cite and analyze evidence, or present an argument in a persuasive manner.) A complex learning outcome can be broken down into component skills, each of which can be examined separately. Each component has a set of graduated categories that define and describe 4 (usually) levels of attainment, from "minimal" to "proficient". Scores assigned to each level help to quantify how students perform on the task, and the results provide direction for future improvement.
Student Learning Assessment (formerly "Inside Assessment") Website
Office of the Provost Student Learning Assessment project gathers
learning goals for courses and programs, curricular maps, assessment
plans, and annual assessment reports for all UW-Madison academic
programs. The public portal, Student Learning Assessment
is a great place to start, with links to templates and guides, the
campus-level assessment toolkit, announcements about workshops; for more
information, consult the Provost's Office colleagues who are listed on
the site's contact page.
Unlike formative assessment (see above), information is gathered at the end of a course or program, and changes made affect future students rather than students in progress.