Measuring Student Knowledge with Video Content

Finding ways to engage students and measure their understanding while they watch video content

Instructional Challenge

Students are engaging in more video content as part of their courses. It is essential for instructors to know not only whether students are watching content but also to understand how they can deliver content that engages students and how well students are learning the content presented in these videos. This document will provide recommended solutions and approaches you can use as you deliver video content to students.

|

Length of Video Content

The first significant barrier to overcome concerning getting students to watch video content and ensure they learn from it is the length of videos. Several articles have studied the ideal length of a video. The ideal length of engagement drew from several factors, including cognitive load, attention span, learning outcomes, and student preferences. The research confirms the recommendation based on the Danforth, Schumacher, & Ma article that found students prefer video content limited to 4-6 minutes (Danforth, Schumacher, & Ma, 2012). Guo, Kim, and Rubin confirmed that video length is the most significant indicator of student engagement, with the highest level of engagement being between 1-3 minutes, the median engagement time around 6 minutes, and engagement dropping off after 6 minutes. Studying engagement levels with instructor-present videos, they found that engagement dropped off after 6-9 minutes, compared to 3-6 minutes with narrated presentations (Guo, Kim, and Rubin, 2014). Pi and Hong found that students who viewed video content with a length of between 4-6 minutes had the best learning scores after watching the video. They also confirmed that mental fatigue begins at 10 minutes and worsens after 22 minutes (Pi & Hong, 2016). Guo, Kim, and Rubin also found that students watching an instructor-present video in which the instructor was seated at a desk with a closer focus on their face engaged twice as long (6-12 minutes) than they did when they watched a lecture capture video (3-6 minutes). Both formats found engagement levels deteriorating after 12 minutes (Guo, Kim, & Rubin 2014).

Recommendation 1

Regardless of how engaging and compelling the content is in a 50-minute video lecture, you can assume that students will struggle to consume it regardless of their interest or desire to succeed in your course. The best approach is to break these videos into smaller 8-10-minute videos. Not only will this increase the likelihood that students watch them to completion, but it will also help focus learning on one concept at a time.

Viewing Analytics

Instructors can view analytics on video content placed in Kaltura. This will provide a list of students who have viewed the video and their completion rate for the video. Be aware, however, that this data is not presented in real time and may take a day to update. Analytics may also not be collected from students running privacy or ad-blocking software. Instructors should not use this information to assign points or prove whether students did or did not watch content.

Measuring Student Knowledge Upon Viewing Video Content

While analytics can tell you how many students completed viewing a video, it cannot tell you how successfully the content facilitated learning, what content students understood, and what content they struggled with. The following approaches can be used to develop a learning pathway that introduces content and provides opportunities for students to apply and measure their understanding of the content while, at the same time, providing you with valuable information you can use to facilitate their learning.

Recommendation 1

The first option is to place pauses in the video for students to stop and reflect on materials before moving on to the next topic. This can easily be done using Kaltura Interactive Video Quiz for your lecture video. This feature should be used only to provide self-assessment of content and not as a formal measure of learning or to assign points for watching the video to completion. Interactive Video Quiz can include the following question types:

  1. Multiple Choice: Questions with only one correct answer. Questions have a 180-character limit for questions and a 140-character limit for answers. 
  2. True / False: Question with the choice of True or False. Questions have a 450-character limit.
  3. Reflection Point: This is a question with no answer. The video will pause and allow you to point out specific items in the video to guide your viewer's attention. This non-graded question will not be part of the quiz score. Questions have a 500-character limit.
  4. Open Question: The student can type in a free text answer. This non-graded question will not be part of the quiz score. Questions have a 200-character limit. Answers have a 270-character limit.

As the instructor, you can view this feedback to see if students understand the content and what questions they may still have.

Recommendation 2

Consider creating a practice quiz that students can complete after the video that measures their understanding of the content presented. Students do not receive a grade for practice quizzes, even though the quiz results display the number of points earned in the quiz. The instructor can see who has completed the exam, however. Communicate to students that the exam is the way you will check their understanding of the content presented in the video. This is a better way of ensuring students have watched the video through the available analytics. This approach also allows students to immediately apply the content presented and quickly identify gaps and misunderstandings in their understanding. To create a practice quiz, follow these steps:

  1. Select Quizzes from the course navigation.
  2. Select the + Quiz button.
  3. Under the Details tab, give your quiz a name, and provide instructions in the text box.
  4. Select Practice Quiz from the Quiz Type menu.
  5. Click on the Questions tab.
  6. Add the questions to your survey. Select from the following question types: Multiple Choice, True/False, Fill In the Blank, Fill In Multiple Blanks, Multiple Answers, Multiple Dropdowns, Matching, Numerical Answer, Formula Question, Essay Question, File Upload Question, and Text (no question).
  7. When done, select Save & Publish.

Instructors can view the results of practice exams in Canvas to get a picture of student comprehension of the content covered in the video.

Narrated Presentations

With regard to the creation of narrated presentations, the recommended tools are Microsoft PowerPoint for creating a narrated presentation to deliver content to students, Kaltura for storing the content, and Canvas for delivering content to students. Kaltura Machine Captioning is also available to make recorded content accessible. Guides can be found for creating narrated presentations for Mac and Windows users.

Other In-context Video Solutions

Tools like Camtasia and Captivate make it easy to create questions in a video or narrated presentation. Using a technology called SCORM, instructors can upload these modules into your Canvas course and present the content to students. While this approach is pedagogically sound, it is essential to know that Canvas currently has issues processing the results of SCORM content, and instructors should not implement SCORM content designed to record results to the Grades tool within Canvas.

Sources/Citations

  • Danforth, S., Cullen, R., and ;Ma Y.J. "Evaluating Format Preferences and Effectiveness of Video Podcasts Related to Nutrition Education and Recipe Demonstrations." Journal of Nutrition and Dietetics, (2012). 112, A19.
  • Guo, Philip, Juho Kim, and Rob Rubin. "How Video Production Affects Student Engagement: An Empirical Study of MOOC Videos." In Proceedings of the First ACM Conference on Learning@Scale Conference. (2014). pp. 41-50.
  • Kizilcec, René. Kathryn Papadopoulus and Lalida Sritanyaratana. "Showing Face in Video Instruction: Effects on Information Retention, Visual Attention, and Affect." Proceedings of The SIGCHI Conference on Human Factors in Computing Systems. (2014). pp. 2095-2102.
  • Pi, Zhongling, and Jianzhong Hong. "Learning Process and Learning Outcomes of Video Podcasts Including the Instructor and PPT Slides: A Chinese Case." Innovations in Education and Teaching International. (2016) 53(2). pp. 135-144.
  • Sweat, Anthony, and Kenneth Alford. "Getting Started with Blended Learning Videos." Faculty Focus. June 2019.
  • Wang, Jiahui, Pavlo Antonenko, and Kara Dawson. "Does Visual Attention to the Instructor in Online Video Affect Learning and Learning Perceptions? An Eye-Tracking Analysis." Computers and Education (2020) 146. 103779).
  • Wilson, Kristen, et al. "Instructor Presence Effect: Liking Does Not Always Lead to Learning." Computers and Education. (2018) (122). pp.205-220.



Keywords:
videos, length, measure, quiz, understanding, knowledge, recommendations, approaches, solutions, Canvas, Kaltura
Doc ID:
104241
Owned by:
Timmo D. in Instructional Resources
Created:
2020-07-22
Updated:
2024-08-23
Sites:
Center for Teaching, Learning & Mentoring