The Learner Activity View for Advisors (LAVA) Overview for Pilot

The Learner Activity View for Advisors (LAVA) is a dashboard that provides visualizations of student access to materials in Canvas courses.

Note: This document describes a learning analytics approach to help support student success.

  • LAVA is currently available as part of a pilot program; if you’re not in the fall 2021 pilot, you will not have access to LAVA. This project is part of UW-Madison's broader efforts around learning analytics. More information about learning analytics at UW-Madison is available from the Vice Provost for Teaching and Learning and DoIT Academic Technology.
  • Remember that students are unique individuals and are much more than their data - be sure to consider the student holistically.
  • Guiding principles for the ethical use of data for learning analytics state that learning analytics should be used to benefit students, instructors/advisors should be transparent about using learning analytics, instructors/advisors should respect students’ privacy and confidentiality, and should minimize any potential adverse impacts.


What is the Learner Activity View for Advisors (LAVA)?

The Learner Activity View for Advisors (LAVA) is a course-level learning analytics dashboard that displays Canvas, Kaltura Mediaspace, and Unizin Engage eText data to academic advisors through a table and three visualizations. LAVA provides aggregate information about how a student is engaging, participating, and performing in online activities in a course.


LAVA interface showing 4 tabs


LAVA allows advisors to see general trends in course activity across four data tabs.

Data TabsThumbnail Image of Data
Canvas Courses Login

Shows the last time students logged in to Canvas, and also the last time a student accessed each enrolled course over the past 12 months.

[data table]
last course access table-thumbnail
Canvas Courses Access & Engagement

Shows the times and days students are accessing their courses.

[heat map]
heatmap_thumbnail.png
Canvas Course Summary Performance

Shows the grade range for each assignment group, the student's performance, as well as the average and the 25th-75th percentile.

[box and whisker plot]
summary performance thumbnail
Canvas Course Weekly Performance

Shows how well a student is doing in a course each week compared to the class average.

[bar and line graph]
weekly performance thumbnail

How can I use LAVA?

LAVA allows advisors to quickly view data visualizations about student activity in their courses. Advisors can look for patterns and ask questions that can support students and potentially improve their course and curricular performance.

The goal of LAVA is to provide information about students's participation, engagement, and performance (PEP) in their courses.

What questions can LAVA help answer?

Participation and Engagement

  • Did a student login to a course?
  • Are they keeping up with their assignments?
  • What days and times are students engaging most with a course?
Performance
  • How is a student doing with graded assignments, discussions, quizzes?
  • How is the student doing compared to their peers in the course? Might they need additional support or resources?
  • Is the student’s performance changing over time - trending up or down?
It is important to note that while LAVA can reveal certain patterns and comparisons, it does not and cannot provide an explanation for those patterns and comparisons.
  • Advisors should never use a single data point from a learning analytics tool to take any major action in regards to a student or course. Data from learning analytics tools can be one factor that helps guide advisor decisions and actions but should never solely dictate them.
  • Learning analytics tools, like LAVA, are simply meant to help provide advisors with additional information and a way to visualize it. It can help you see patterns of behavior and help to identify areas where you might want to reach out with more questions to a student or instructor.
  • Instructors have academic freedom to use/or not use Canvas and other instructional tools in a way that best supports their pedagogical approaches. Course activities and student participation are not always tracked consistently or digitally, especially for residential, on-site courses.

What learning analytics data can I access in LAVA?

LAVA combines and presents data from several learning technology tools:

  • Canvas - learning management system used to post course materials and activities
  • Kaltura MediaSpace - for media in the course (video, audio, images)
  • Unizin Engage eText; an e-text reader for online textbooks

Important notes about the data:

Please keep in mind that LAVA data is aggregated from multiple sources that are updated at various frequencies, depending on the tool. Data can take up to five days to appear in LAVA. While visualizations are not showing information from the current day, they still provide a unique view and early insights into engagement trends and student activity.

  • Canvas page views capture when students are clicking on resources within their Canvas courses. 
    • Think of page views very broadly as students participating or engaging with a course as they’re clicking around and interacting with course content.
    • Because the page view data is based on requests to the server, the numbers for page views may be greater than what we traditionally think of as a page view, for example if a student loses their connection or reloads/refreshes a page. As a result, page view data should be used as a good approximation to student activity and not an absolute metric. 
    • This data is most valuable when seeking to understand if activity did occur, and as a means of comparison across students within a course or when viewing trends week to week.
  • Kaltura Mediaspace counts anytime a student clicks on a Kaltura video/audio/images embedded in Canvas through the Canvas-Kaltura integration
    • Instructors can embed or link media into their Canvas courses using several methods. For example, if a student clicks on a Kaltura video embedded in a Canvas course using an embed code, that will not be counted.
  • Unizin Engage eText counts anytime a student loads or reloads a Unizin Engage eText page. 
    • It cannot track off-line reading, if a student downloads the content as a PDF. 
    • Unizin Engage eText data does not include any data generated by other publisher Digital Learning Tools such as Cengage Mindtap or MacMillan Launchpad.
  • Please see the Official Data Definitions for the Learner Activity View for Advisors at the end of this document for more information about the data in LAVA.

How do I access LAVA?

LAVA is currently being piloted by a small group of advisors.

  • It can be accessed at https://tools.canvas.wisc.edu/lava
  • Follow the instructions on the screen (provide your NetID and password, and use MFA/Duo for authentication) to access the tool.
  • If you cannot access LAVA, you are not part of the current fall 2021 semester pilot.

Launch & Student Lookup Page

The LAVA launch page takes you to a Student Lookup page where you can search for a student by first/last name or by NetID. Searches must be 3 or more characters, and if a student has never been in Canvas, they will not appear in LAVA.

  • You should only search for your own students, per FERPA regulations. (Note: As with most of our campus IT tools, access monitoring and logs are in place. As part of our process, those logs are periodically reviewed to ensure appropriate data is being accessed.)
  • As part of our commitment to transparency, all students and instructors involved in this pilot have been informed about the use of learning analytics/LAVA.

search student and select dashboard


Once you have searched for a student, you will be given dashboard links for students matching your search criteria.

  • Click on the red Dashboard button to view the student’s dashboard, which includes a table and 3 visualizations.
  • If more than one student matches a user’s search criteria, select the corresponding Dashboard link for the student you are advising.

Dashboard Navigation

After you have selected a student’s dashboard, you will be able to select any of the visualizations from the four tabs across the top of the page. The first two tabs provide participation and engagement data, while the last two tabs provide data about student performance.

  • Canvas Courses Logins shows the last time students logged in to Canvas, and also the last time a student accessed each enrolled course over the past 12 months.
  • Canvas Course Access & Engagement show the times and days students are accessing their courses.
  • Canvas Course Summary Performance shows how a student is performing relative to their course peers, by assignment grouping (eg. Readings, Quizzes).
  • Canvas Course Weekly Performance shows how well a student is doing in a course each week compared to the class average.
dashboard navigation using four tabsThe first two tabs provide participation and engagement data, while the last two tabs provide data about student performance.

 

Filters and Default Views

  • Each tab has different filters, depending on the data.
  • The filters on each tab function independently; for example, if you select a specific course on one tab, when you navigate to a different tab and want to see information about the same course, you will have to re-select the course.
  • Default views also vary across the different tabs, based on the data that’s available.

Canvas Courses Login

On the Canvas Courses Logins page, you can see a list of the courses a student has been enrolled in for the last 12 months.
  • You will also see Last Canvas Login at the very top of the page, which is the most recent time a student logged into Canvas. This could be to access a non-credit course, for example SOAR or AlcoholEdu.
    • Note that all times are converted to Central Time.
  • The Last Course Activity column shows the most recent time a student logged into specific courses over the past 12 months.
  • You can filter by Term, Course (alphabetically) and Last Course Activity (the default shows earlier course activity first, and later course activity last).
  • The filters in this table allow you to sort and organize data, versus drilling down to more specific data.

Canvas courses logins and class list

What questions can this view help answer?

  • Is the student participating online in a course? 
  • Is the student active (online) in all of their courses? 
  • When was the last time the student was online in Canvas? 
  • What courses are the student enrolled in? What courses did the student take last term?

Canvas Courses Access & Engagement

The Canvas Courses Access & Engagement visualization is a heat map that shows when a student is accessing courses in Canvas. You can filter by Term, Course, Date, and Activity Type (eg. Canvas Quiz or Kaltura Video) for a student.

The purpose of this visualization is to see when a student has the most interactions with a particular Course or Activity Type, visualized by Time of Day and Day of Week. This is especially useful when you want to see the time of day or week that a student is interacting with Canvas the most.


heatmap of when students are most active in course

You can look at engagement across multiple courses or just one course. In this example, the student has the most interactions with this course in the evenings, especially Mondays, Wednesdays and Fridays. 


How to read this heat map

  • The darker red color indicates more interactions (‘hotter’); whereas lighter red and grey colors indicate fewer interactions (‘cooler’). 
  • A blank area or white space indicates that a student has not interacted with a particular Canvas course (or courses) during that time.
  • Click on any of the cells/boxes in the visualization to see detailed information showing how many views (interactions) during that particular time and day. 
  • Interactions capture the total number of times a student has loaded an Activity Type in Canvas (essentially clicking on an activity in Canvas) during that particular hour and day of the week.
  • Keep in mind this is showing all of the interactions across all the Mondays, Tuesdays, Wednesdays, etc, for the date range. If a student is highly engaged with Canvas during a particular time, this could be a good time to reach out to a student, since they’re more likely to be engaged with course material during times of high interactions.
  • All times are converted to Central Time; (international students may appear to be online in the course at unusual times).
  • If you’re not seeing data/all the cells/boxes are white:
    • The visualization will automatically default to the most recent semester, so keep in mind that especially during the beginning of a term, if a student hasn’t logged into Canvas yet, you may see very little or no data (typical for the beginning of a semester). As the term progresses, more data will appear.
    • If a student was not enrolled in any course during a specific term (ie, Fall 2020-2021), or is not yet enrolled and actively using Canvas during a particular term, you will not see data for that student (all the cells/boxes will remain white).

How to filter this heat map

  • This visualization also defaults to include all Courses and Activity Types - you can deselect and/or multi-select using Shift/Click. 
    • This is the only visualization that will display multiple courses.
  • Selecting the Term, Courses, Activity Types, and Data heat map filters will allow you to drill down and *only* see data for a particular term, course, activity type, and date range.
  • Activity Types will vary across courses, depending on how an instructor has designed and organized their course (eg. by Canvas Pages or Canvas Modules) and the types of content, assignments, and activities that have been incorporated into their online Canvas course.
  • Additionally, you can select a date range if you are interested in seeing a student’s engagement with Canvas for a particular week or month. Use the calendar tools to select the start and end dates.

Handy Tips

  • Students will start/stop accessing each course based on their schedule and when the class starts and stops meeting. If you’re comparing activity and dates between courses for the same semester, you might notice that start and end dates for each course will vary (based on when the student accessed courses).
  • If a student drops a course during the term, it will still appear in the Courses and you can see the first/last times the student accessed that course.
  • The colors are relative for each student and course; so the actual numbers of interactions for the darkest red blocks will vary.
  • The color blocks do not indicate duration; for example if a student interacts or clicks on one page and then navigates away immediately, it will still show as an interaction on the heat map.

What questions can this view help answer?

  • When is a student active in a course?
  • Are there general patterns of student activity across all courses in a semester? ?
  • Has a student increased/decreased the amount of time they’re generally spending in courses?
  • When might be a good time to reach out to a student?

Canvas Course Summary Performance

The Canvas Course Summary Performance visualization shows how a student is doing in a Canvas course compared to their classmates. This particular visualization organizes data by assignment group (i.e. homework, quizzes, attendance, etc) using “box and whisker” visualizations. (Please also check out the Canvas Course Weekly Performance visualization on the last tab, which is similar but shows a student’s performance compared to their classmates each week.)


Canvas course summary performance box and whisker plotIn this example, the student (yellow circle) has more points (is performing higher) than the class average (the gray line in the red box) for all of the course activities. 

 


How to read this box & whisker plot

  • This visualization shows box and whisker plots for each assignment group, allowing you to see how a student is performing grade-wise in their course compared to their classmates while also seeing the course grade distribution, all at once.
  • In the screenshot above, you can see that this student is doing well compared to their peers in each assignment group for this particular course. This student is doing especially well compared to their classmates in the “exam” category.
  • The yellow circle shows where a student falls in the overall course distribution for each assignment group.
  • The red box shows the middle 50% of overall Canvas course performance across all students taking the course, and is derived using the lower 25th% and upper 75th% quartile values.
  • The class average for each assignment group appears as a horizontal line in the center of the red box.
  • The whisker (vertical line)shows how many points that students in the course have earned for an assignment group. 
    • When you click on a box and whisker plot you can see the total range of points (from zero to the maximum number of points) that are allotted for a given assignment group. 
    • Note: the whisker may start above 0 (if every student earned some points in that assignment group) and extend above the maximum points (to reflect extra credit).
  • The light grey bar at the top of each box and whisker plot shows how many points are available for each assignment group.
  • Additionally, more information can be obtained by clicking on any single box and whisker plot. You can see the max points, class high, class average, class low, and, importantly, the student’s score here. (Please refer to the data definitions if you need more information about the data contained in this popup box.)
  • If you’re not seeing data:
    • This visualization also defaults to showing the most recent or upcoming term, and may appear blank at first unless a term with Canvas course activity is chosen.
Clicking on the assignment group displays more detailed information. This example shows the "whisker" vertical line is not touching the baseline; the lowest score for this assignment group is 18 points, higher than the 0 baseline.

 

How to filter this box & whisker plot

You can filter by Terms or Courses in this visualization, and when filtering, *only* data for the term or course chosen will be displayed.

Handy Tips

  • This visualization is highly dependent on how an instructor organizes their Canvas course (for example, if they do not separate tests into a separate category, you may see quizzes and tests combined; likewise discussions and participation assignments could be seen together).
  • There’s a lot of information provided in this view, but at a glance, you can see that the higher up the student score (the yellow circle) appears in the distribution, the better the student is doing, compared to their classmates in that assignment group.
  • Conversely, the lower down that the student score (the yellow circle) appears in the class distribution, the worse the student is doing compared to their classmates in that assignment group.
  • This is a fast way to check to see how a student is doing compared to their classmates, and you may see that a student who is doing very well on homework assignments may not be doing as well on quizzes or exams, for example, or vice versa.

What questions can this view help answer?

  • How is a student doing generally, across all of their courses? Are they in-line with their peers?
  • Are there some types of learning activities where a student really excels?
  • Are there some activities where a student might need additional support or resources?
  • How is a student doing with graded assignments, discussions, and quizzes?

Canvas Course Weekly Performance

The Canvas Course Weekly Performance visualization is a bar and line graph that shows how a student is doing in their Canvas course, compared to their classmates; this visualization organizes the data weekly.
weekly performance shows the student compared to the rest of the classThe student's performance in this example trended up and then held steady for the rest of the semester.  

 

How to read this bar & line graph

  • You can see the class average grade as a percentage by week (the red bar) compared to a student’s grade (also as a percentage) by week (the yellow line).
  • The student’s grade shown is cumulative over time (i.e., in week 4, the grade shown is that student’s grade up to week 4).
  • You may also click on any weekly bar to see the exact grade percentages for both the class weekly average and the student’s grade during that week.
  • If you’re not seeing data:
    • This also defaults to showing the most recent term and may appear blank at first unless a term with Canvas course activity is chosen.

How to filter this bar & line graph

You can filter by Terms or Courses in this visualization, and when filtering, *only* data for the term or course chosen will be displayed.

Handy Tips

  • If a student’s grade improves, or, alternatively, starts to trend downward over time, you can see that easily in this visualization and could be an indication that a student may be struggling or thriving in a course over time.
  • Note: assignments can be created in Canvas with no due date, in that case they will appear in the first week.

What questions can this view help answer?

  • How is a student doing, compared to their classmates?
  • Does this student appear to be keeping up with assignments?
  • Is the student's performance changing over time - trending up or down?

Official Data Definitions for LAVA

LAVA official data definitions are reviewed and approved through a data governance process; these definitions are sometimes rather technical; please review the information provided about each visualization (above). These data definitions are also published on the Glossary of Terms Used in Administrative Dashboards and Reports page on the Office of Data Management & Analytics Services website.

The definitions are listed below, and organized into where they appear in LAVA.

Definitions across several visualizations

Assignment Group (LAVA):

Type of Canvas activity classified by Canvas course instructor or administrator and can include: Announcements (Canvas)*, Assignments (Canvas)*, Collaborations (Canvas)*, Discussions (Canvas)*, Gradebook Grades (Canvas)*, Modules (Canvas)*, Pages (Canvas)*, Quizzes (Canvas)*, or other groupings that an instructor decides to use.

Course(s):

This is a unit of instruction approved through faculty governance. *Assignment Group (LAVA): Type of Canvas activity classified by Canvas course instructor or administrator and can include: Announcements (Canvas)*, Assignments (Canvas)*, Collaborations (Canvas)*, Discussions (Canvas)*, Gradebook Grades (Canvas)*, Modules (Canvas)*, Pages (Canvas)*, Quizzes (Canvas)*, or other groupings that an instructor decides to use.

Term(s):

A period of instruction: Fall, Spring, or Summer.

Canvas Courses Logins Definitions (table)

Last Canvas Login (LAVA):

The most recent time a student logged any Canvas course, including non-credit courses (non-course specific) *Last Course Activity (LAVA): The most recent time a student logged into the Canvas course that corresponds to the row and course listed in the table displayed.

Last Course Activity (LAVA):

The most recent time a student logged into the Canvas course that corresponds to the row and course listed in the table displayed.

Canvas Courses Access & Engagement Definitions (heat map)

Activity Types (LAVA): Note that this can also be referred to as Assignment Group

Type of Canvas activity classified by Canvas course instructor or coordinator and can include: Announcements (Canvas)*, Assignments (Canvas)*, Collaborations (Canvas)*, Discussions (Canvas)*; Gradebook Grades (Canvas)*, Modules (Canvas)*, Pages (Canvas)*, Quizzes (Canvas)*, or other groupings that an instructor decides to use.

Day of Week (LAVA):

Day of week in Central Time, including weekends.

Heatmap Filters (LAVA):

Filter used for narrowing down data by Term(s), Course(s), Dates, and Activity Types. This filter defaults to the most current term but users can select any term in the most recent 12 months.

Interactions (LAVA):

Total number of times a student has loaded an Activity Type in Canvas; can be filtered by term, course, Activity Type itself, and date range.

Time of Day (LAVA):

Hour of the day in Central Time

Canvas Course Summary Performance Definitions (box & whisker)

Assignment filters(LAVA):

Filter used for narrowing down data by Term(s) or Course(s). This filter defaults to the most current term but users can select any term in the most recent 12 months.

Class Average (by Assignment group, LAVA):

The arithmetic average or the sum of all Canvas course points available in each assignment group divided by the count of numbers in the Canvas course assignment group *Student Score (by Assignment group, LAVA): The grade shown as points that a student is currently assigned in a Canvas course assignment group. 

Class High (by Assignment group, LAVA):

The maximum number of points any student has currently been assigned in a Canvas course assignment group.

Class Low (by Assignment group, LAVA):

The lowest number of points any student has currently been assigned in a Canvas course assignment group.

Course Points (LAVA):

Total Canvas course points available for all students, including extra credit points.

Max Points (by Assignment group, LAVA):

The maximum number of points available in a Canvas course assignment group for all students.

Total Range (LAVA):

All points available in a Canvas course within an assignment group for all students, including extra credit points 

25th - 75th Percentile (LAVA):

This center area represents the middle 50% of overall Canvas course performance across all students taking the course, and is derived using the lower 25th% and upper 75th% quartile values.

Canvas Course Weekly Performance Definitions (bar & line graph)

Class Average (Weekly, LAVA):

The arithmetic average or the sum of all Canvas course points available in each week divided by the count of numbers in the Canvas course week, shown as percent.

Student Score (Weekly, LAVA):

The grade shown as percentage a student is currently assigned in a Canvas course week

Week (LAVA):

The week number of a Canvas course; week 1 generally corresponds to the first week a course begins.

Weekly Filters (LAVA):

Filter used for narrowing down data by Term(s) or Course(s). This filter defaults to the most current term but users can select any term in the most recent 12 months.

Where can I find more information about learning analytics?

For additional information on using learning analytics at UW-Madison, check out:

Support

For any issues, questions or concerns with LAVA or how to use it, please contact LearningAnalytics@office365.wisc.edu.




See Also:




Keywords:LAVA, learning analytics, guiding principles, data, FERPA, Data governance, advisors   Doc ID:112677
Owner:Kari J.Group:Learning Analytics
Created:2021-07-27 11:18 CDTUpdated:2021-08-19 17:43 CDT
Sites:Learning Analytics
Feedback:  0   0