InterPro - Teaching & Learning - Theories & Principles - Assessment & Evaluation
EPD Instructional Resources
Assessment
A note about assessment. With any course or module, the learner should be assured that they are accomplishing the desired outcomes throughout the course. In fact, with most forms of learning there is a need to allow the learner and instructor to see and understand how learners are progressing throughout the course. This is done with assessment, and it is useful to build some form of assessment into courses or modules. There is no right or wrong assessment strategy, but there are various ways of attempting to determine what a student knows and is able to do, and so it is important not to think of assessment as the final stage of a course, but to build it into the course using all of the types of assessment (diagnostic, formative, and summative) where appropriate.
There are many different types of assessment that can be completed as part of an instructional unit, both online and off-line, and individually or in teams. Here are some examples:
- Exercises that are completed online (multiple choice questions, drag and drop exercises, matching exercises)
- Essays and project reports
- Discussions online or in class
- Case studies, individually or in groups
- Applying knowledge through simulation
- Applying knowledge in a realistic setting
- Formal written examinations which can contain a variety of question formats
- Reflection exercises
- ePortfolios
When creating learning assessments, it is important to remember to align assessments with learning objectives/goals and instructional activities. In other words, an instructor needs to ensure they are assessing what is being taught, and in the way it is being taught. Therefore it is useful to begin developing assessments along with the learning objectives and goals, and to iteratively refer back when creating activities, course materials, as there will be additional tuning of objectives and assessment to make sure they are in alignment.
As a general rule, there are 9 qualities of an effective assessment (adapted from Reeves, 2011):
- Closely aligned with learning objectives and activities
- Provide clear, simple and detailed instructions
- Clear audience and purpose (explain why this is important)
- Valid
- Reliable
- Gives ownership (and sometimes choice) to the learner
- Achievable by learners (is the assessment at the proper level
- Checklist or Rubric to help learners understand expectations
- Assessment plan or instrument for validation, in order to ascertain whether learning has occurred
One final note, on the role of assessments to help assure management that employees have been trained adequately in areas where there may be a legal requirement, such as induction training, and occupational health and safety training. Assessment and evaluation can help to provide a record that such training has been carried out successfully.
Evaluation
Kirkpatrick's 4-level model of evaluation is a widely used model for evaluating instructional programs and products around the world. The two most popular levels are level 1 and 2. It is important to note that the degree of difficulty in measuring and evaluating grows in difficulty as on progresses through the levels, this is, in party, why levels 1 and 2 are often most common and popular, but are also probably most useful.
Level 1. Reaction
This is an evaluation of the learner's experience of the e-learning, and is normally collected using a questionnaire. Rather than just questioning whether they liked the training, ask the following questions:
- Did they find the objectives relevant?
- Did the course maintain interest?
- Was it easy to navigate?
- Was enough support provided? If not, what was missing?
- Was there a balance between interactivity and information?
- Was it valuable to them?
- Was it transferable to the workplace?
- Do not make the questionnaire too large (maximum 25 questions), and design it to collect qualitative and quantitative data. Use a small ranking scale (e.g. 1-5 or 1-10) for most questions, and have a few open-ended questions. Track this evaluation over time to aid in continuous improvement.
Level 2. Learning
This level measures learning results. Did the trainee actually learn the knowledge, skills and attitudes the e-learning was designed to teach? For more quantitative approach make sure to collect a pre-training baseline by running a pre-training test and compare it to the post-training test results. The impact that the training had can then be clearly measured. Measurements at this level might indicate whether the instructional methods used are effective or not, but they do not provide information on whether the trainees actually changed their behavior back on the job.
Level 3. Behavior/Transfer to the Job
This level measures the trainee's ability to transfer what they have learned back to the job. Did their behavior change? Many would argue that this level truly measures the success of the training. Performance indicators will need to be measured, such as:
- The time it takes to complete work
- The level of defects in product development
- Damage or injury caused by accidents
- Customer satisfaction
- Enlist the help of supervisors, operational managers and customers to define and collect the data (observation surveys and production data)
- It is useful to collect baseline measures for these performance indicators (and these may be useful in building a business case) or, if feasible, use a control group who have not participated in the online training. Allow time after the training for behavioral change to take place.
Level 4. Business Results
This level evaluates the impact of the training on business results. It may be very difficult to evaluate the impact of a single course on the company bottom line but the impact on business performance levels will be measurable, such as in reduced complaints. If sales training is used as an example, it may be possible to measure changes in sales volume, customer retention and profitability after the training has been implemented. Profitability may also indicate a lower cost to produce a product a deliver a service, because of development process improvements due to technical, quality, management and safety training.
Level 5. Bringing it All Together
A fifth level of evaluation has been suggested by many - that is bringing it all together in a return on investment (ROI) calculation.
Selected Resources
- Instructional activities: Classroom Assessment Techniques (CAT)
- Effective assessment
- A Primer: Diagnostic, Formative and Summative Assessment
- Authentic assessment overview
- What is authentic assessment?