Teaching and learning (T&L) data is an asset that can be used to allow educators to make informed decisions about curriculum design, student engagement, and instructional strategies. If accurate, accessible, and actionable, T&L data can support learning in a myriad of ways, engage instructors in continuous improvement practices, and enhance decision-making creating meaningful educational experiences.
Unfortunately, usable data does not just happen. Intentional, proactive thought is necessary from early on in the course design stage to ensure meaningful data is generated during a course. Luckily, there are some simple principles that can be adopted that have outsized impact.
The usage and effectiveness of T&L data is closely tied to course design. This document outlines intentional course design practices aimed at maximizing the potential of teaching and learning data. It includes detailed scenarios where these practices can be applied and provides recommendations or resources for effective implementation.
It is important to remember that these recommendations are based solely on generating usable, impactful data. Instructors, instructional designers, and other educators should incorporate these recommendations with discretion based on their own course and student needs.
The recommendations below were developed from suggestions gathered during a cross-functional multi-department working session, evidence in literature, and informed by an analysis of course data.1
The Four Recommendations
Focus on consistent, complete and clear gradebook practices
Scenario: Using due dates and total points to enhance student performance
Challenge: While teaching an introductory course, Professor Badger notices that some students are consistently submitting assignments late, which makes it difficult to track their progress and identify those who might need additional support.
Solution: Professor Badger decides to input due dates and total points possible for each assignment in Canvas in order to collect consistent data on when assignments are submitted and how students are performing.
Benefits:
- Assignment submissions are collected in a timely and consistent manner
- Students can be identified who frequently submit late, miss deadlines, or score low for targeted intervention
- Students can manage their time better and understand the importance of each assignment
- Instructors can identify which assignments are scored low for large portions of the class
- A wide variety of reporting/analysis can be accurately applied to the data to support decisions
Effective grade book practices are essential for accurate data collection and analysis. Instructors should set clear due dates and total points possible for all assignments. Grade weighting should be properly outlined within the course, to ensure that each assignment's weighting is accurately reflected in the overall course grade.
Recording assignment weights accurately is essential for analyzing student performance and course outcomes. It ensures data reflects each assignment’s impact on final grades, and helps identify areas needing support. Specifying total points for assignments aids students to understand their contributions to final grades, enhancing time management and prioritization.
To maintain the integrity of data, instructors should assign zeros only after due dates have passed, rather than prematurely. This approach prevents misleading data that could distort analytics results. Finally, centralizing all grades in a single gradebook allows educators to monitor student progress accurately and intervene when necessary. For example, all grades for the course should be in one Canvas course, even if you have multiple Canvas courses, such as one course for the overall course and one for group work.
To accurately set up your gradebook, please review Learn@UW’s Canvas gradebook tutorial video, “Canvas: Gradebook”, or the Gradebook Manual (written version).
Focus on descriptive and decipherable naming conventions and labels
Scenario: Consistent naming conventions for clarity and prioritization
Challenge: As students begin to settle into a new course, they notice inconsistencies in Canvas, leading to confusion. The students report difficulty finding resources in Canvas due to inconsistent naming conventions. Some assignments are labeled by week, others by topic, and a few by due date only. For instance, one assignment is called "Essay 1," another is "Week 6 Assignment," and a third simply says "Paper." A similar issue is reported on the resources and files in the course, with some titled by the content while others are titled by the name of the author(s). This inconsistency causes confusion, leaving students unsure of which tasks to prioritize, if they've completed all necessary work, or if all resources were reviewed.
Solution: Professor Badger implements a consistent naming convention for all course content, such as "Week 4 - Research Paper" or "Module 3 - Quiz on Linear Equations." Each assignment and resource title now includes relevant details, making it easier for students to locate the materials they need and stay on track with deadlines.
Benefits:
- Students can easily find assignments and resources for the corresponding week, module, or topic, reducing confusion and improving task prioritization.
- Student performance across the course can be assessed in a streamlined process, helping identify areas where students are struggling or excelling.
The clarity and organization of course data are enhanced by using consistent naming conventions and accurate labels for all activities and resources within Canvas. For example, using specific names like "Week 4 Quiz - Thermodynamics" or “Module 4 Quiz - Thermodynamics” helps students, educators, and analysts easily identify and locate materials.
Instructors should use a standardized naming convention for all course content, such as including the week, module, topic, or type of activity in each name. This makes the materials easier to find and the data easier to analyze.
With clear and consistent naming conventions, T&L data can more easily provide valuable insights into student engagement with course materials, track performance across course activities, assess the effectiveness of instructional strategies, and identify areas where students may be struggling or succeeding.
Be intentional about content organization
Scenario: Optimized course layout to improve student engagement
Challenge: Midway through the semester, many students have reported difficulties navigating the Canvas course. They are struggling to find assignment instructions, which are scattered across the site, and often encounter outdated or broken links. Content is not organized in any logical manner. This disorganization is affecting their ability to effectively complete tasks on time or understand which resources align with which topic, module, or week. A review of the course layout reveals that the structure is cluttered and disorganized, with too many links, including several from previous semesters that no longer work. Essential resources are also deeply nested within the course, making them difficult to access.
Solution: Professor Badger streamlines the course by removing outdated links, consolidating important materials, and implementing a consistent organizational structure. Simplifying the navigation will significantly improve students' ability to find and complete their assignments efficiently.
Benefits:
- Students may increase engagement with course content since they can more easily locate and access materials.
- Students can prioritize course activities better and manage time better.
- Instructors can identify which modules, week, or topic (depending on organization structure) students are engaging with the most/least
The way content is organized within Canvas directly impacts both the student experience and the accuracy of data collected for analytics. To ensure that students can easily navigate the course and find the materials they need, educators should organize content into clearly defined modules based on activity type, such as writing assignments, quizzes, or discussions.
To implement this effectively:
-
Check for broken links as links sometimes stop working over time, or when copied over from another course page. Broken links can cause an inflated number of clicks as students repeatedly try to access the resource. This weakens the integrity of the data generated from the link and may lead instructors to mistakenly believe the resource is valuable. In addition, broken links have a negative effect on user experience and accessibility.
-
Avoid redundant links: Link to each resource or activity only once. Similar to broken links, redundant links can overinflate student usage and lead to inaccurate metrics.
-
Use Canvas’ built-in features: Categorize and structure content logically by module, week, or topic. For example, grouping all assignments or resources related to Thermodynamics into a single module titled Module 4 - Thermodynamics helps streamline access and reduces confusion. This provides data on which course content is related and how certain materials are being used in reference to others.
-
Use thoughtful naming conventions and labels: Follow the guidelines described in the previous recommendation to ensure clarity and consistency.
-
Consider other Canvas course design guidelines and strategies - many schools and colleges have local resources and checklists; the Center for Teaching, Learning and Mentoring has a collection of course design strategies for Canvas courses.
Additionally, hiding and/or disabling unused links in Canvas’ navigation ribbon helps reduce confusion and unnecessary navigation throughout the course. For example, remove the Announcements link from the navigation ribbon in Canvas Settings if you are not using Canvas Announcements. This not only allows a better learner experience, it also removes unnecessary navigation options, ensuring cleaner data.
Maintaining clean and well-structured courses is essential not just for enhancing the learning experience but also for ensuring that the data collected is accurate and meaningful. A well-organized course layout allows for more reliable tracking of student engagement and performance, which in turn enables better data analysis. This organization helps educators identify trends, monitor student progress, and pinpoint areas that may need improvement.
Mindfully integrate third-party applications and tools
Scenario: Analyze video engagement to improve quiz performance
Challenge: Professor Badger has observed that students' quiz scores, particularly those linked to Kaltura video content, have been declining. He suspects this might be due to poor engagement with the videos but needs to confirm this. Specifically, he wants to understand how students are interacting with the videos—whether they are playing them fully, and if so, which students are not engaging with the content as expected.
Solution: Professor Badger can access detailed analytics from videos that are embedded correctly. These metrics include the number of times each video is played, the number of unique viewers, total minutes watched, and the average video completion rate.
Benefits:
- Helps to identify which videos are being watched, or not.
- May help to identify students who are not watching videos or are not watching the entire video. (Professor Badger has learned that pop-up blockers can impact Kaltura analytics, and students sometimes view content together - so those possibilities also need to be considered.)
- Analyze video effectiveness to better support students
- If students are frequently re-watching certain videos or skipping others, it may indicate areas where content is not clear or where additional scaffolds may be necessary .
- Helps identify which types of videos (e.g., lecture recordings, tutorials, animations) are most effective
Incorporating third-party applications and tools within Canvas through LTI (Learning Tools Interoperability) integration is crucial for centralizing all course-related data in one place, making it easier to track and analyze student interactions and performance.
Applications and tools are added into Canvas in several ways. However, integrating the LTI directly into Canvas is the most effective way for collecting actionable data. For example, correctly embedded LTIs can provide better data on who the user is, what role they have, and the action that was recorded for any particular event that occurred within Canvas.
Implementing this practice will involve different steps depending on the specific tool being integrated. Here is a comprehensive resource that details the integration process for each tool type and includes step-by-step guides, best practices, and troubleshooting tips to ensure successful integration; it also lists who to contact for more support with each tool. Following these guidelines allows instructors to effectively embed tools like Kaltura Media, TurnItIn, and other content directly into Canvas, enhancing the overall learning experience and improving data collection for better insights into student engagement and performance.
Embedding content within Canvas improves learning analytics by centralizing data, giving educators a comprehensive view of student engagement and performance. It also enhances the student experience by providing seamless access to resources, maintaining data integrity, and supporting effective decision-making. This unified approach streamlines access and ensures accurate analytics.
Instructors should prioritize Learn@UW centrally supported tools or those supported by your departmental resources when selecting teaching and learning tools.
- The Learning Analytics Center of Excellence analyzed data from a random sample of 40 courses, each with at least 20 enrollees, across multiple semesters and confirmed that the recommendations discussed enhance data quality from Canvas courses, resulting in more impactful data. For more information on this analysis contact LearningAnalytics@office365.wisc.edu.
