The Learner Activity View for Advisors (LAVA) Overview

This document describes the Learner Activity View for Advisors/Canvas Flags (LAVA), which is a new resource accessible via the Advising Gateway.

During the Spring 2024 term a learning analytics pilot is being conducted with the intent of providing academic advisors with access to and awareness of trends across the course array of a student in two broad areas: performance and participation.

The “Learner Activity View for Advisors” (LAVA) resource will be made available to the pilot academic advisors via the Advising Gateway. Advisors will have no direct access to Canvas courses or any associated material.  LAVA exposes only high-level trend indicators of patterns across a student’s enrolled course array. Communications about the pilot to pilot students and to the instructors whose courses they are enrolled in for Spring 2024 will be sent out in January.

Some of the major goals of the LAVA pilot include:

  • Evaluating this approach to learning analytics for advisors (providing awareness of broad trends ACROSS a student’s enrolled course array)
  • Understanding the utility and efficacy of this approach and of the LAVA resource
  • Working with academic advisors to determine what interventions/actions could look like based on insights provided by access to this new information
  • Understanding the impact of access to learner activity data on advisors (bandwidth, impact of seeing flags, ability to engage/intervene, etc.)
  • Assessing whether this insight into learner activity is helpful to advisors in their work
  • Assessing whether this insight into learner activity proves helpful to students (getting ahead of potential bad outcomes, improving outcomes, etc.)

Notes: 

What is the Learner Activity View for Advisors/Canvas Flags (LAVA)?

The Learner Activity View for Advisors/Canvas Flags is an awareness function that displays performance and participation data to authorized users through a new capability in the Advising Gateway.

search for a student in Advisor Gateway - view a Canvas Flag

Search for a student - display shows Canvas Flags (see detailed image, below)

If a student has an active Canvas flag, a checkmark will appear in the Flags column in the search results on the Advising Gateway. The Flags column and checkmarks will appear on all of the Gateway’s three search tabs: All Students, My Advisees, and Last Viewed.

Clicking on the Flags column heading will sort the search results into two groups; students with active flags will be listed first (in alphabetical order), followed by all students without active flags (in alphabetical order).

search for a student on advisor gateway - view Canvas flag

Canvas Flags details on search page

By incorporating these data directly into the Advising Gateway, it offers authorized users a view of student academic activity and performance during the active term, within the normal workflow and familiar Advising Gateway interface.

student profile showing Canvas Flags tab with active flags indicator

Student profile page - showing Canvas Flags tab with the active flag indicator

Features:

  • This is a learning analytics approach, using course activity and gradebook data from Canvas (and other integrated tools) and accessed within the Advising Gateway. More details about the data, and when performance and participation flags will appear are provided below.
  • Data appear both in a Flags column in Advising Gateway search results, and in a new Canvas Flags tab on the Student Profile page within the Advising Gateway. If low performance or participation trends are present across multiple courses (relative to their peers within those specific courses), a flag will appear in that tab on the student profile page.

  • The flags may help an advisor start a conversation with a student, if there is a potentially concerning trend or pattern of behavior across multiple courses. 

About the Flags

Two types of flags may appear:

  • Participation - course activity
  • Performance - grades

To view flag data and details, click on the tab to show the accordion menus that are labeled with the flag names. 

Canvas Flags tab - with no raised flags

Canvas Flags tab - with no raised flags

If flags are present, there will be a label telling you which flag has been raised. In the example below, the Performance flag has been raised. 

Performance flag has been raised

In the example below, there are two flags raised - both Participation and Performance flags have been raised. 

Both Participation and Performance flags have been raised

Flagged Courses

When the flag menus are expanded, you can see the list of courses the student is in, and which specific courses contribute to the active flag. 

  • Course lists will appear under both the Participation and Performance sections, since there may be a flag raised in one area but not in the other, for the same course. (For example, a student’s performance is not yet flagged, however they have not participated in Canvas for the last five days, compared to their peers, for several courses.) 
  • Courses that meet the criteria in the flag details will appear in the Flagged Course column.  All other courses will appear in the No flag or no data column. 

Canvas Flag details example - showing course lists

Canvas Flag details example - showing courses

Two or More Courses - Looking at Trends

LAVA/Canvas Flags is designed to show trends or patterns of behavior across a student's array of enrolled courses.  A student generally must have two or more courses that meet the criteria in the flag details for the flag to be “raised”. 

Caveats: 
  • If a student is only enrolled in one course, a flag may be raised if the threshold is met in that one course. 
  • If a student is enrolled in two courses, a flag will be raised if the threshold is met in only one course. 

Flag Details

The flag details box describes the course data that is used to determine if courses have flags. 

Important reminders to keep in mind: 

  • Instructors teach using many different strategies and pedagogical approaches. How a course is set up, and how much an instructor uses Canvas and other linked tools for teaching activities will impact the data that can trigger these flags. 
  • The more an instructor uses Canvas and other linked tools, the more useful Canvas Flags may be.
  • Don’t assume anything based on just one data point; remember to validate and ask questions. This is a starting point to help with a conversation.  

Participation Flag Details (Course Activity)

Student activity in the course is compared with other students enrolled in that course. Participation data uses activity in Canvas, along with several other tools if they are integrated within the Canvas course (more details below). 

  • Participation is measured as any activity in an online tool in the course.
  • Wherever possible, students are compared to the other students enrolled in all sections of a course. 

What is Participation in a Course? 

Canvas participation is based on page views in a course, along with student activities. Student activities may include:

  • Accessing a course page
  • Announcement - posting a new comment
  • Assignments - submitting an assignment
  • Commenting - posting a new comment on an announcement or discussion
  • Collaborations - viewing and/or editing a collaborative document
  • Conferences - joining a web conference
  • Discussions - posting a new comment
  • External tools that are linked within the course (see more info, below)
  • Pages - creating a page 
  • Quiz - starting a quiz, submitting a quiz
More About External Tools

Other tools may be linked within a Canvas course; for example Kaltura MediaSpace, Piazza, TopHat, Pressbooks, Engage eText or Zoom are other Learn@UW tools that instructors use. 

  • When a student opens an external tool from within the Canvas course, that participation will be counted.
  • Activity occurring within external tools is NOT generally captured.

Activity in other online tools that are not connected to Canvas - or are not launched from Canvas - will NOT be included as participation for flag calculations. For example, a course that uses a standalone WordPress website for course activities won’t show participation data. 

A participation flag will be raised if:

  • The student is 5 days behind the median of their peers in aggregate participation. In other words, they are well behind their peers in accessing the course activities. 
  • There must be at least two courses that the student hasn’t logged into for five days. (If a student is only enrolled in one course, a flag may still be raised if they’ve not interacted with that one course.)

How the Participation Flag is Calculated

  • LAVA calculates the median most recent activity time for all students in the course.
  • If the students’ Last Activity DateTime is 5 or more days before the median, the flag may be raised (if this occurs across two or more courses, when the student is enrolled in more than one course). 
  • This flag will be most useful for courses where many students are frequently logging in (for example, they all take a quiz synchronously in a live class).
  • If the student does not have a last activity date (has never interacted with the course), and the course start date is set, the students’ last activity date defaults to the course start date. 
  • If the course start date is not set, the default start date is the start of the term.   
Additional Details
  • Peers for the activity quartiles are based on the course the student is enrolled in Canvas. This means if sections have been split across multiple courses (for example, the discussion sections have been moved to their own course, separate from the lectures) they may not be compared with all their peers in the course.

Performance Flag Details (Grades)

Performance data uses the Canvas Gradebook. It uses the current calculated grade for a course. 

  • The more assessment and grading activity that is handled in Canvas, the more useful this flag will be.
  • Grades are included based on the grade posting policy set in Canvas
  • Grades can be included from external tools, if they are integrated within the Canvas course (e.g. Top Hat, Atomic Assessments, ScanTron, Kaltura Quizzes.)
  • Grades can be manually entered or imported into the Gradebook.

What is Performance in a Course? 

  • Performance is measured as any and all grades entered in the Canvas Gradebook.
  • Wherever possible, students are compared to all sections of a course

A performance flag will be raised if:

  • A student is a full letter grade (8 percentage points) behind the bottom quartile of their peers. 
  • Students are scoring in the bottom quartile - their grade performance is in the lowest 25th percentile for the course. 
  • There must be at least two courses that meet the performance threshold (If a student is only enrolled in one course, a flag may still be raised.)

How the Performance Flag is Calculated   

  • LAVA calculates the bottom quartile grade (the lowest 25th percentile) for a course.
  • It then subtracts a single grade (e.g. from A to B) which is 8 percentage points.
  • If the students’ grade is less than the above calculation, a flag may be raised (if this occurs across two or more courses, when the student is enrolled in more than one course)
Additional Details
  • If the Gradebook is not turned on, nobody gets flags (for example, a face-to-face course that doesn’t really use Canvas). 
  • Peers for grades are based on the course and enrollments as originally defined in the Student Information System (SIS). If different sections are later split apart (e.g. the discussion sections have been moved to their own course, separate from the lectures) and grades are given in different sections, we will get duplicate grades for the same SIS course.  In this case, the maximum score from the two sections will be used, as there is no way programmatically to determine how an instructor will later combine these grades before final grade submission. We default to the maximum score to reduce the risk of raising a flag unnecessarily.
  • Instructional choice about grading (and activity for that matter) is a personal choice of each instructor; consistent grading practices should not be assumed.

For additional information on using learning analytics at UW-Madison, check out:

For any issues, questions or concerns with LAVA or how to use it, please contact LearningAnalytics@office365.wisc.edu.



KeywordsLAVA, learning analytics, guiding principles, data, FERPA, Data governance, advisors, canvas flags, CCAS, Cross College Advising Services   Doc ID123619
OwnerKari J.GroupLearning Analytics
Created2023-01-27 16:45:04Updated2024-01-08 14:31:33
SitesLearning Analytics
Feedback  0   0