Are my students viewing my videos
Instructors often assign students the task of watching a video before class with the intent that this content will prepare them for class activities. This content can be a lecture, narrated PowerPoint, demonstration video, or interviews/case studies. When students do not complete this assignment or when after viewing the content they have not acquired the knowledge or understanding at the desired levels, this can cause problems for both the instructor and student. Being unprepared often leads to failed classroom learning experiences, an unplanned readjustment in-class activities, and falling back to lecturing on the content covered by the video content.
This document will cover several approaches you can use to monitor students' compliance with watching video content. It will also provide ways of determining whether compliance equals preparation for planned classroom activities.
|
1) Identify the question
Monitoring compliance vs. measuring understanding
Once you have delivered content in ways that generate user data, the next step is to determine what question you want to use to answer. If you want to know whether students watched the video before class, that falls under a monitoring compliance action. If you want to know whether watching the video prepared students for future learning, that is a measuring understanding action. These two approaches have slightly different solutions.
Monitoring compliance
You can review the analytic data provided by Kaltura MediaSpace to gain considerable insight into student behavior. You can determine:
- Whether individual students viewed content,
- Whether they completed viewing content,
- How frequently they reviewed content, and
- Which parts of the content they reviewed.
Be aware, however, that there are no perfect solutions. These approaches provide a more informed view of student behavior but not a complete one. The information provided needs to be viewed and understood within a certain context.
2) Identify data collection process
Identifying data sources for compliance
The first step in the process of monitoring student compliance with watching video content is to deliver it in a way that provides you information on their viewing. This can shed light on whether and how completely students have engaged in the content, and possibly, how well they have understood that content. While there are several ways to deliver media to students in order to collect data in viewing/comprehension, this document focuses on the most common and simplest approach — an embedded link in a Canvas page to media housed in Kaltura MediaSpace. Using the Kaltura Embed Media tool within Canvas, you can upload existing media files into Kaltura MediaSpace, embed content into a Canvas page, and collect a variety of learning analytic data on the viewing behaviors of that content from students.'
3) Implement data collection
3) Data collection for compliance
Uploading and embedding video into a Canvas page
Kaltura MediaSpace is UW-Madison's media streaming service and is integrated with Canvas. Kaltura media can be embedded wherever Canvas allows you to use the rich content editor.
Note: to embed an existing Kaltura Mediaspace item in a Canvas course, please see Canvas - Embedding an Existing Kaltura MediaSpace Video in a Canvas Page [UW-Madison]
We recommend instructors new to using media with their Canvas course start here. Uploading and embedding Kaltura media in a Canvas Page or other Canvas tool is an easy way to make your Kaltura media available to your students. Embedded media provides more context and keeps the student in Canvas rather than sending them to an external link outside of Canvas. It also allows Kaltura to collect more detailed analytics.
- In your Canvas course, go to the Canvas page, discussion post, or quiz discussion you want to embed Kaltura media in. Click Edit.
- Move your cursor to the point in your Canvas item where you want to insert Kaltura media.
- Click the Embed Kaltura Media button which looks like the colorful Kaltura logo. If your browser is not of sufficient width you may need to click the three dot button on the right to expand the menu to see the button:
- A window titled Kaltura Media will open showing the media content associated with your Kaltura account.
- To upload media at this point click Add New and select Media Upload to upload a new video to your account:
- Read and click the "I agree to the above terms and conditions" disclaimer.
- Scroll down to click + Choose a file to upload or drag your MP4 file onto the window:
- A window will open allowing you to select the video file to upload. While Kaltura can process a wide range of video formats, we find MP4 videos generally work well. We recommend 1280 x 720 video as a good balance between file size and quality. We also recommend you try to keep your video files under 2 GB - the larger the file the longer it will take to upload and process.
- Your file will upload. While it uploads you can edit the name, description, and tags.
Upload Speed
Even when Kaltura MediaSpace is running smoothly, other internet service providers may experience slower service, which could limit your upload speed.
- Upload speeds and Kaltura's upload performance varies, but an upload speed of 10 Mbps should be able to upload a 250 MB file to Kaltura in about 4.5 minutes.
Processing
The amount of time it takes Kaltura MediaSpace to process a file depends on file size, duration, quality, and usage.
Generally it takes about 2-3 times the video's duration for Kaltura MediaSpace to display your video in low definition (high definition versions take longer to process). For example, a 15 minute video will usually take about 30-45 minutes to process (in low-definition). When Kaltura MediaSpace is being heavily used, processing times will lengthen and could take up to 24 hours.
If your video takes longer than 24 hours to process, please call the DoIT Help Desk. More detailed information is available in Kaltura's document involvingtranscoding times and quality of service.
- If you want to customize how your video is displayed, see the Editing Media Settings section in this document. Otherwise, select the </>Embed button to add the media.
- After embedding, you will see an icon indicating it is still processing. Once done processing the video will show up on your Canvas page. You can continue to edit the page and add text if you'd like. Once you're done click the Save button:
- Canvas will display your page with your embedded Kaltura video:
Editing Media Settings
You can edit your file name, description, and tags as it uploads. After your file is done uploading you can:
- Edit the file's name, description, tags, and add collaborators. More information on the collaboration feature is available in Kaltura - Media Collaboration: Changing Media Ownership, Adding Co-editors, Co-publishers, and Co-viewers [UW-Madison] .
- Save button on the bottom will simply save your changes to the metadata while your video uploads.
- </> Save and Embed button in the upper right will embed your video using the default settings in your Canvas Page or other area the Canvas rich content editor is available.
- Gear icon to edit your embed settings. An Embed Settings window will display:
- Choose Player: Choose your player based on the type of media you are embedding. As of 8/15/24 all players automatically include a transcript widget. You do not have to select a specific Transcript player. Video users can choose between:
Default player - This player is setup with the most commonly used options. Most users use this player. The download icon may appear to allow for download of closed caption or attached files, but the source video file can not be downloaded.
Download player - Use the download player if you want users to be able to download the original source media file locally to their computer. This can be useful if you have students without reliable high speed internet connections. They could go to a library or coffee shop to download your media to view later. If you choose the download player, viewers can download the original version of the video you uploaded to Kaltura by clicking the Download button in the upper right of the video: - Max Embed Size: Users can choose how large the media is displayed within the Canvas rich content editor.
- Thumbnail Embed: Turning thumbnail embed on renders an image that will display instead of the video player. The student needs to click on the image to load the player. Generally we recommend keeping this option turned off.
- Auto Play: Turning auto play on plays the media immediately when the student loads the page with the embedded media. We recommend you keep auto play off and let the student choose to play the media by clicking the "Play" button.
- Set as my default settings for media type: Video (or Audio, or Image): If you like the settings you have selected you can check this box to make future embeds use them without having to set them every time.
- Start at: Set the start time for the media item.
- End at: Set the end time for the media item. Note that users can still play the entire video!
- </> Embed: When you have set the options you want, click the </> Embed button to embed the media.
If after embedding your Kaltura media you decide you'd like to change the options you selected it is easiest to delete your embedded media from the page and re-embed.
- Choose Player: Choose your player based on the type of media you are embedding. As of 8/15/24 all players automatically include a transcript widget. You do not have to select a specific Transcript player. Video users can choose between:
Embedding an existing Kaltura video into a Canvas page
Kaltura MediaSpace is UW-Madison's media streaming service and is integrated with Canvas. Kaltura media can be embedded wherever Canvas allows you to use the rich content editor. We recommend instructors start here if they already have media in Kaltura MediaSpace
Note: to upload and embed an new Kaltura Mediaspace item in a Canvas course, please see Canvas - Upload and Embed a New Kaltura Mediaspace Video in a Canvas Page [UW-Madison]
Note: You can also embed other media sources, such as Youtube videos. Please see Instructure's Canvas Community site for instructions on adding other video sources, including Youtube media.
Embedding Kaltura media in a Canvas Page or other Canvas tool is an easy way to make your Kaltura media available to your students. Embedded media provides more context and keeps the student in Canvas rather than sending them to an external link outside of Canvas. It also allows Kaltura to collect more detailed analytics.
- In your Canvas course, go to the Canvas page, discussion post, or quiz discussion you want to embed Kaltura media in. Click Edit.
- Move your cursor to the point in your Canvas item where you want to insert Kaltura media.
- Click the Embed Kaltura Media button which looks like the colorful Kaltura logo. If your browser is not of sufficient width you may need to click on the three dot button on the right to expand the menu to see the button:
- A window titled Kaltura Media will open showing the media content associated with your Kaltura account. By default it is on the My Media tab. These directions assume you are embedding a video you already uploaded. You can also:
- Upload a new video (see above)
- Embed a video from a course media gallery: If you are using a course media gallery, first click on the Media Galleries drop-down menu at the top of the screen and click on your media gallery. More information on using media galleries is available in Kaltura - Using "Kaltura Gallery" Course Media Galleries and Analytics in Canvas [UW-Madison]
- If you are embedding a piece of media you already uploaded, scroll to or search for the media you want to embed. Click the </> Embed button next the media you want to embed:
Optional: If you would like to customize how your media is embedded in Canvas, instead of clicking the </> Embed button, click the gear button:
- An Embed Settings window will display:
- Choose Player: Choose your player based on the type of media you are embedding. Video users can choose between the "Default Player" and "Download Player". Both players automatically include a transcript widget if a closed caption file is available.
- Max Embed Size: Users can choose how large the media is displayed within the Canvas rich content editor.
- Thumbnail Embed: Turning thumbnail embed on renders an image that will display instead of the video player. The student needs to click on the image to load the player. Generally we recommend keeping this option turned off.
- Auto Play: Turning auto play on plays the media immediately when the student loads the page with the embedded media. We recommend you keep auto play off and let the student choose to play the media by clicking the "Play" button.
- Set as my default settings for media type: Video (or Audio, or Image): If you like the settings you have selected you can check this box to make future embeds use them without having to set them every time.
- Start at: Set the start time for the media item.
- End at: Set the end time for the media item.
- </> Embed: When you have set the options you want, click the </> Embed button to embed the media.
If after embedding your Kaltura media you decide you'd like to change the options you selected it is easiest to delete your embedded media from the page and re-embed.
- Choose Player: Choose your player based on the type of media you are embedding. Video users can choose between the "Default Player" and "Download Player". Both players automatically include a transcript widget if a closed caption file is available.
- The video will show up on your Canvas page. You can continue to edit the page and add text if you'd like. Once you're done click the Save button:
- Canvas will display your page with your embedded Kaltura video:
Embedding a YouTube video into a Canvas page
You can "save" YouTube videos into your Kaltura MediaSpace account and embed them into your Canvas page. Only public videos from YouTube can be embedded. While this feature does not download the video and requires that the original YouTube video exists, it provides a few major advantages:
- It removes advertisements.
- You can modify the title and description.
- You can embed student reflection prompts within the media.
- You can collect the same student engagement analytics as your own media.
- It allows for professional captions to be added to meet ADA requirements for accommodations.
Directions
- In your Canvas course, go to the Canvas page in which you want to embed the YouTube media. Click Edit.
- Move your cursor to the point in your Canvas item where you want to insert Kaltura media.
- Click the Embed Kaltura Media button which looks like the colorful Kaltura logo. If your browser is not of sufficient width you may need to click on the three-dot button on the right to expand the menu to see the button:
- Select +Add New, then select YouTube.
- Enter the Video Page Link or the Content ID and click Preview. To obtain the URL, right-click on the video on YouTube and copy the URL.
- Click Preview to edit the details including name, description, and tags.
- Click Save.
4) Review the data
Review data for measuring compliance
There are a few important caveats about the data.
- Instructors should be aware that the information Kaltura MediaSpace presents is not real-time data. It may take some time for the data to be reported. Take this into consideration before you plan to assign points or call people out for non-compliance.
- While you have access to individual viewing data, be aware that Kaltura MediaSpace analytics may not be able to collect a student's identity if they are running privacy or ad-blocking software. They would need to configure their software to allow stats.kaltura.com and analytics.kaltura.com domains access. The viewing results would be collected but aggregated into an Unknown user category.
- Individual user data only captures the user who clicked on the link. If students are studying together, for example, only the student logged into Canvas would be recorded.
Viewing Media Analytics
- Add the MyMedia tool to your Canvas sidebar by clicking Settings from the Canvas navigation menu. On the Settings page, click the Navigation tab. Location MyMedia from the list of hidden tools. Click on the three-dot icon to the right of the tool. Click Enable.
- Select the MyMedia from Canvas navigation bar.
- Click on desired media entry.
- Choose Analytics from the ACTIONS drop-down menu.
Option 2:
- Go to Kaltura MediaSpace (mediaspace.wisc.edu)
- Log into the service by selecting MyMedia. You will be prompted to log in with your NetID.
- Click on desired media entry.
- Choose Analytics from the ACTIONS drop-down menu.
There is a lot of information provided by Kaltura MediaSpace if you embed your content within a Canvas page. This document will walk you through several reporting details. View a full list of reporting data. View a list of terminology used in reporting.
Media Views
Directly below the video, you will see a graph of users (both general and unique) views of the video over time (based on the timespan identified at the top of the report.
User Engagement
Given the timespan you select specify in Kaltura MediaSpace, you get the following data:
- Player impressions — how many times has a page with a link to your video been viewed?
- Plays — how many times has the video player started to play the video?
- Unique viewers — how many individuals have started to play the video?
- Minutes viewed — how many total minutes has the video been played by users?
- Average Completion Rate — what percent of people watched the entire video?
Engagement by User
Directly below the Media Views graph, you should see the option to View Engagement Per User. With content being embedded in a Canvas page, each user's viewing behavior can be tracked. The data for each student provided includes information not only on the number of views but also an analysis of which parts of the video they watched and how often they watched it. This report shows:
- Not viewed — this section wasn't viewed by the user.
- Viewed once — this section was viewed once by the user.
- Viewed twice — this section was viewed twice by the user.
- Viewed more than 2 times — this section was viewed more than two times by the user.
Media performance over time
This report provides information based on the timespan defined above for the following details:
- Number of player impressions
- Number of plays
- Number of unique viewers — number of authenticated viewers, all anonymous viewers will be counted as a single viewer
- Number of minutes viewed
- Average completion rate — how much of the video was watched
- Average drop-off rate — measured by viewer reaching playback quartiles
Player Engagement Levels
The engagement funnel shows the student's level of engagement with the entire video during the identified timespan of the report. This can be a way to quickly check to see the depth of students' engagement with the content (keeping in my the limitations of the data). The following metrics are shown in the funnel:
- Player impressions
- Plays
- Plays that reach <x%> percent play through — this can be selected by the user.