How do I know whether my students read the content I assigned?

Using a variety of approaches to determine student readiness after reading assigned content

Problem Statement

Instructors often assign students the task of reading content before class with the intent that this content will prepare them for class activities. This content can be a Canvas page, an article, a handout, or other text-based content. When students do not complete this assignment or when after viewing the content they have not acquired the knowledge or understanding at the desired levels, this can cause problems for both the instructor and student. Being unprepared often leads to failed classroom learning experiences, an unplanned readjustment in-class activities, and falling back to lecturing on the content covered by the reading content.

This document will cover several approaches you can use to monitor students' compliance with reading content. It will also provide ways of determining whether compliance equals preparation for planned classroom activities.

|

Delivering content to students using Canvas

Canvas Modules and Pages allow instructors to organize content to help control the flow of the course and collect analytics on content. Modules are used to organize course content by weeks, units, or a different organizational structure. Modules essentially create a one-directional linear flow of what students should do in a course. Each module can contain files, discussions, assignments, quizzes, and other learning materials. Module items can be added to the course from existing content within the modules.

Link to article or document

linked doc

While there are several ways to provide access to a document, providing access to the document through the Modules tool is the most common way. The format of linked documents can include anything as long as the student has the application that supports the viewing of the file. The most typical file formats for linked course readings are either Adobe PDF or Microsoft Word. To link to an article or document:

  1. In Course Navigation, click the Modules link.
  2. To add an item, click the Add Item icon.
  3. Select File from Add drop-down menu.
  4. If the file you wish to link to is already in your Canvas course, select the desired file.
  5. If the file you wish to link to needs to be uploaded, select [ Create File(s) ] then select Choose Files button. Specify the location within Canvas to save the file from the Folder drop-down menu.
  6. Determine any desired level of indentation within the Modules tool.
  7. Click Add Item.

This will create a link to the file within the Modules tool. When clicked, the browser will display the file within Canvas if the format is supported. If not, the file will be downloaded to the student's computer.

Deliver content within a Canvas Page

Canvas Page with text

There are times in which you have content in text format and want students to read it within a Canvas page, instead of linking to a document.

If you want to create a new Canvas Page, follow the directions below:

  1. In Course Navigation, click the Modules link.
  2. To add an item, click the Add Item icon.
  3. Select Page from Add drop-down menu.
  4. If you wish to create a new page, select [ Create Page ] then name the document in the Page Name box. Note: If you are going to want to track usage, provide a descriptive name to make it easier to identify it in the analytic reports.
  5. Determine any desired level of indentation within the Modules tool.
  6. Click Add Item.

Embed article in a Canvas Page

embedded doc

If you need to embed a document within a new page, follow the directions below:

  1. In Course Navigation, click the Modules link.
  2. To add an item, click the Add Item icon.
  3. Select Page from Add drop-down menu.
  4. If you wish to create a new page, select [ Create Page ] then name the document in the Page Name box. Note: If you are going to want to track usage, provide a descriptive name to make it easier to identify it in the analytic reports.
  5. Determine any desired level of indentation within the Modules tool.
  6. Click Add Item.

If you want to embed a document within an existing document, follow the directions below:

  1. In Course Navigation, click the Modules link.
  2. To add an item, click the& Add Item icon.
  3. Select Page from Add drop-down menu.
  4. Select the document to add to your module.
  5. Determine any desired level of indentation within the Modules tool.
  6. Click Add Item.

Once a page is added to the Modules tool, click on the link and click Edit.

Embedding a PDF link

  1. Place the cursor where you want to create the link.
  2. Click on the arrow next to the link tool and select Course Link.
  3. If the file exists within Canvas, select Insert from the menu, then either Upload Document to add a file from your computer, Course Document to link to a file in your course files, or User Documents to link to documents in your own user folder.
  4. Once the link appears on the page, click on it once and choose Link Options.
  5. Choose Preview inline, and Expand preview by Default to display the PDF within the Canvas Page.

Monitoring compliance vs. measuring understanding

Once you have delivered content in ways that generate user data, the next step is to determine what question you want to use the data to answer. If you want to know whether students engaged with the content before class, that falls under a monitoring compliance action. If you want to know whether students' engagement with the content prepared them for future learning, that is a measuring understanding action. These two approaches have slightly different solutions. We will start with basic monitoring.

Monitoring compliance

How do I know whether my students read the content I assigned?

The goal of monitoring compliance may have a different meaning from one instructor to another. The data available within the Learner Engagement Analytics Dashboard (LEAD) regarding compliance with an assigned reading may fall short of expectations. For the sake of this document, compliance is simply the measurement of whether is student did or did not click on the link to a reading, as well as how many times they did click on the link.

LEAD

LEAD is a course-level learning analytics dashboard that displays Canvas, Kaltura Mediaspace, and Unizin Engage eText data to course instructors using Tableau, our institutional data visualization tool. Data can be viewed as an aggregate of the entire course, or at the level of an individual student. The LEAD Page Views by Activity Type visualization can be used to help or contribute to answering the question, "Has the student viewed the content I assigned?".

This visualization allows you to see how many times your students are accessing the various activity types in your course. You can use filters to drill down and see how much a specific piece of content in your course is being accessed, and who is accessing it. This visualization can also give you a sense of how many students and which students are accessing specific pieces of content in your course. For example, you can select a specific course reading from Main Activity Type, and then select one specific assignment (eg. Twigg Reading) from Name of Activity to see how many times students in your course clicked on that specific assignment; you can also filter by a student to see which students accessed a specific item

LEAD Page View By Activity Type

Note: This document describes a learning analytics approach to help support student success.

The LEAD Page Views by Activity Type data could help offer insights into students' access of course materials. This information may be useful to you to find out: 

  • Which documents, media, or activities students are accessing. 
  • The amount of student access to various activity types, for example Kaltura Videos or Canvas Discussions
  • The amount of student access to specific course materials, such as an individual content page or video.

What data are available in LEAD?

Campus tools such as Canvas, Kaltura MediaSpace (video/audio/images), and Unizin Engage eText are connected to student roster information. This allows student data to be connected with a record of their course access and interaction, such as:

  • Course pages or videos they’ve clicked on
  • Grades stored in the Canvas gradebook
  • Participation with activities such as assignment submissions, or discussion posting
  • Times of access

More information about LEAD is provided in the Learner Engagement Analytics Dashboard Overview KB doc, including more details about the data and the official data definitions.

How to access LEAD

LEAD is currently available for instructors teaching for-credit courses who are enrolled in Canvas as a principal instructor, auxiliary instructor, or supervisory instructor.

Instructors can access LEAD at go.wisc.edu/lead. Follow the instructions on the screen to log in. 

Once inside LEAD you will have access to a home page and three visualization pages. 

  • Page Views by Date and Hour 
  • Grades by Page Views 
  • Page Views by Activity Type

For easy access to other learning analytics resources, add the Learning Analytics for Instructors Widget to your MyUW page.

Data visualization format: Bar Chart

The visualization format is a Bar Chart

  • Horizontal bars represent quantitative data, the lengths of the bars will vary, depending on how many times an activity is accessed. For example the longer bars reflect that students are accessing that content more than the shorter bars.
  • Depending on the types of content and activity you have in your course, as well as how you have your course setup, you will see different bars and counts displayed here. (Note: for this example, numerous courses were selected from the LEAD admin interface, the data displayed is aggregate data across many courses. This screenshot is not typical of what one instructor might see for one course, after one month of activity.)
  • For example in the screenshot below, the instructors may be using few Discussions, resulting in a relatively low count of Page Views.
  • On the other hand, there is a much higher Page View count of Canvas Quizzes indicating more students access this activity type.
  • Notice there's not a horizontal bar/Page Count for Engage E-Text; it suggests that this activity type is not used in the courses that were selected for the example.

LEAD Page Views by Activity Type screenshot

Filtering the data

The default view is to show the Page View count data for all students in a course, counted from course activities from the start of the semester to within 5 days of the current time. (LEAD data is aggregated from multiple sources that are updated at various frequencies, depending on the tool.) 

This may be good to see general trends of access to the course materials.

If you want to see the Page View counts for a specific piece of content, or for an individual student, you can use the filtering functions on the left of the screen.

If you want to see the data for a particular week, for a category of activity types, for a specific document, or for selected students, you can use the filtering functions available on the left. If you're teaching more than one course this semester, you'll also be able to filter by course name.

  • Changes you make using any of the filters on one tab, will persist as you navigate to the other tabs.

Filtering by date range

You can filter the data to a date range by choosing a start date, end date, or both.
  • Click on the start and end dates to launch the calendars; it's easier to select dates from the calendars than using the scrollbar.
  • The Page View counts will adjust according to the selected dates. This may be useful for reviewing access trends by time.
LEAD Page Views by Activity Type - filter by date

Filtering by activity type and specific activity

You can filter by Main Activity Type. The list in the drop-down will vary depending on how you have your course set up. For example, if you don't use Engage eText or Kaltura Videos, those options will not appear in the Main Activity Type drop-down.

  • The default is All --> Uncheck All and then select one or more Main Activity Type --> Choose Apply. (You may have to click away/off the visualization after you apply the filter.)
  • This step is optional when you are interested in a specific activity but it makes it easier to filter the choices in the next step when you filter by the Name of Activity. Filtering first by Main Activity Type will help you more quickly locate a specific activity (without having to scroll through a list of all your course Announcements, Discussions, Kaltura Videos or other types of content.
  • This example shows filtering on Kaltura Videos.
  • Notice the Page Views By Activity Type count is the same as in the screenshot, above (215,127).

LEAD Page Views by Activity Type - filter by Activity Type

You can drill down to a specific activity using the Name of Activity filter.

  • In this example, the Main Activity Type that was selected is Kaltura Videos so only video titles will be listed in the Name of Activity drop-down.
  • The list of items in the drop-down will reflect how you have named files, pages, assignments etc. in your Canvas course. Use consistent naming conventions and logical file names to help you locate a specific item; for example Mod-2 Video or Discussion Week 2
  • The default is All --> Uncheck All and then select one or more Name of Activity --> Choose Apply. (You may have to click away/off the visualization after you apply the filter.) 
  • This example below shows the data filtered to a single Kaltura Video (the name of the video has been obscured), and notice how the Page View count has now changed (198).

LEAD Page Views by Activity Type - filter by specific Activity

Filtering by student

If you want to check on a student of interest, use the Student Name filter.
  • The default is All --> Uncheck All and then select a Student Name --> Choose Apply. (You may have to click away/off the visualization after you apply the filter.)
  • You can select more than one student.

LEAD Page Views by Activity Type - filter by student

Using the data

[Doc 105064 content is unavailable at this time.]

Caveats and reminders when using learning analytics data

[Doc 105063 content is unavailable at this time.]

Understanding the data

There are a few important caveats to point out here.

  1. Viewing a page simply tells you whether the student accessed the page with the reading. It is the lowest level of compliance an instructor can use — Yes, they clicked on the link or No, they did not click on the link.
  2. Keep in mind that there may be other ways in which a student access the content. Working in a group, for example, might lead to only one student viewing the reading and then sharing it with the other student(s).
  3. LEAD does not show duration, it shows frequency and time. We know instructors are interested in knowing how much time students are spending on course activities, but at this time that’s not available in LEAD. Keep in mind that duration can be tricky to calculate; students may click on a course resource or launch a video and walk away to grab a sandwich or jump to a different tab in their browser.

How can I use this data in a responsible manner?

UW-Madison Learning Analytics Guiding Principles state that learning analytics should be used to benefit students; instructors should be transparent about using learning analytics; instructors should respect students’ privacy and confidentiality, and should minimize potential adverse impacts. The following are examples of valid and invalid uses of the data presented through the LEAD Page Views by Activity Type visualization:

responsible use of data
Valid use of data Invalid use of data
  • Seeing whether most students click the link.
  • Seeing whether any student clicked the link yet.
  • Seeing how many times a student clicked on a link.
  • Assigning points to those who clicked the link.
  • Calling out those who did not click the link.
  • Looking at the data right before class to determine student preparedness.

Measuring understanding

How can I know whether students have achieved competency at this level?

Unlike measuring compliance, measuring understanding requires that you look at things differently. LEAD does not provide any information that would help identify the level of student understanding of content. This would have to be done through the development of a variety of activities aimed at the desired level of competency (ex. remember, understand, apply, analyze, evaluate, or create).

bloom's taxonomy

REMEMBER

REMEMBER — Retrieve, recognize, and recall relevant knowledge from long‑term memory.

  • What do you remember about __?
  • How would you explain __?
  • Describe what happens when __.
  • Find the definition of __.
  • Identify, locate, or name __. 

Possible approaches

  • Canvas Quiz — Quiz questions that have students define terms, label elements, list concepts, or recall points of conclusions.
  • Canvas Discussion — Student post in the Discussion tool a response to a prompt that requires them to recall or retrieve the main points in the assigned content.
  • In-class Discussion — Use TopHat to have students respond to a prompt that requires them to recall or retrieve the main points in the assigned content.
  • Focused Listing — Students are given a single term, name, or concept and are directed to list several ideas that are closely related to the topic.
  • Background Knowledge Probe — Students answer a short survey to collect feedback on their prior learning, including knowledge or beliefs that may hinder or block further learning.

UNDERSTAND

UNDERSTAND — Construct meaning from oral, written, and graphic messages through interpreting, exemplifying, classifying, summarizing, inferring, comparing, and explaining.

  • How would you sort/classify/categorize __?
  • Please tell me what you mean when you say __.
  • Use your own words to summarize __.
  • What is another way of saying __?
  • Compare/contrast __.
  • How can you explain __?

Possible approaches

  • Canvas Quiz — Students are given questions in which they must describe, explain, or identify elements of content.
  • Think/Pair/Share — Students solve problems aloud and try their reasoning on a listening peer. Ask them to answer some of the questions from above individually, share their results with their partner, then report their shared results to the class.
  • Empty Outline — Students are given an empty or partially completed outline of an in-class presentation or homework assignment and a limited amount of time to fill in the blank spaces.
  • Memory Matrix — Students fill in a two-dimensional diagram used to organize information and illustrate relationships in a way that can be quickly analyzed by the instructor.
  • Categorizing Grid — Students sort subordinate terms into the correct categories on a grid.

APPLY

APPLY — Carry out or use a procedure through executing or implementing.

  • Which factors would you change if __?
  • What questions would you ask of __?
  • Which approach would you use to __?
  • What actions would you take if __?
  • What would the results be if __?
  • Why does __ work?

Possible approaches

ANALYZE

ANALYZE — Break material into constituent parts, determine how the parts relate to one another and create an overall structure or purpose through differentiating, organizing, and attributing.

  • Is __ based on fact or opinion?
  • Explain what must have happened when __.
  • What conclusions can you draw from __?
  • What is similar/different from __?
  • What is the motive/underlying theme of __?
  • What is the relationship between __?

Possible approaches

  • Pro and Con Grid — Students follow a decision-making process by reviewing an issue, creating a list of pro and con arguments, and making a decision based on an analysis of those points.
  • Defining Features — Students categorize concepts according to the presence of (+) or absence of (–) critical defining features.
  • Send-a-Problem — Students solve a problem as a group and pass the problem and solution to a nearby group who does the same, with the final group evaluating the solutions.
  • Think-Aloud Pair Problem-Solving — Students solve problems aloud and try their reasoning with a listening peer.
  • Categorizing Grid — Students sort subordinate terms into the correct categories on a grid.

EVALUATE

EVALUATE — Make judgments based on criteria and standards through checking and critiquing.

  • How would you explain __?
  • What changes to __ would you recommend?
  • Do you agree with the outcome of __? Why?
  • What evidence supports your view?
  • What are the consequences of __?
  • How could __ be improved?

Possible approaches

  • Case Studies — Students review a written study of a real-world scenario and develop a solution to the dilemma presented in the case.
  • Structured Problem-Solving — Students follow a structured process to solve problems.
  • Think-Aloud Pair Problem-Solving — Students solve problems aloud and try their reasoning with a listening peer.
  • Pro and Con Grid — Students follow a decision-making process by reviewing an issue, creating a list of pro and con arguments, and making a decision based on an analysis of those points.

CREATE

CREATE — Put elements together to form a coherent or functional whole; reorganizing elements into a new pattern or structure through generating, planning, or producing.

  • What changes could you make to revise __?
  • What theory can you propose for __?
  • How many ways can you __?
  • What might be a solution for __?
  • Develop a plan/proposal for __.
  • Devise your own way to __.

Possible approaches

  • Case Studies — Students review a written study of a real-world scenario and develop a solution to the dilemma presented in the case.
  • Think-Aloud Pair Problem-Solving — Students solve problems aloud and try their reasoning with a listening peer.


Keywordsknowledge, check, pre-class, preparation, article, lecture, online, reading, analytics, chapter, handoutDoc ID122285
OwnerTimmo D.GroupInstructional Resources
Created2022-11-03 12:19 CSTUpdated2023-03-13 10:58 CST
SitesCenter for Teaching, Learning & Mentoring
Feedback  0   0