Learning Analytics Approaches and Teaching & Learning Tools
Introduction
Many of the tools in the Learn@UW suite of tools offer some sort of learning analytics capabilities. Ideas for how to use the analytics across the different learning analytics categories are provided here. Keep in mind, these are just a few ideas and this is not comprehensive. You can think creatively and use many other approaches and methods to obtain data that can provide insightful data to support student success.
Please remember that learning analytics offer a way to visualize patterns of behavior and activity, but cannot provide explanations for those patterns or comparisons. Data from learning analytics tools can be another tool in your toolkit, and can help guide decisions and actions but should never solely dictate them. UW-Madison has approved guiding principles that focus on benefiting students; being transparent about the use of LA; honoring privacy and confidentiality around the use of LA, and minimizing adverse effects.
About the Learning Analytics Functional Taxonomy
The Learning Analytics Functional Taxonomy considers the question, “why are people using learning analytics?” It describes various approaches and is organized into six categories. There are additional use cases and examples for each of the six approaches on the website, including some examples using Learn@UW tools. These are the six categories:
- Access learning behavior
- Evaluate social learning
- Improve learning materials and tool
- Individualized learning
- Predict student performance
- Visualize learning activities
Using analytics from the Learn@UW suite of tools
More details and examples appear below this summary table, which lists six approaches for using learning analytics, and then indicates whether a specific tool might be useful for each approach.
- Brief notes about each approach are provided below. For more complete descriptions about a category, please refer to the LA Functional Taxonomy.
- Check out the KnowledgeBase for more information about the Learn@UW suite of learning tools and analytics that you can leverage in each tool.
| Learning Analytics Functional Taxonomy Approach |
Other Tools |
||||
|---|---|---|---|---|---|
|
yes |
yes |
yes |
yes |
Piazza Pressbooks |
|
| yes |
yes |
Piazza TopHat (Feedback Fruits - College of Engineering pilot) |
|||
|
yes |
yes |
yes |
yes |
all |
|
|
yes |
yes |
yes |
other digital learning tools (publishers' textbooks and interactive homework tools) |
||
|
‘what if’ grade in student view |
|||||
|
yes |
yes |
yes |
yes |
Atomic Assessment |
Access Learning Behavior
Note: Most tools capture user activity and might be useful to view to see whether students' are engaging with course content and activities.
Potential questions for this approach:
- Are my students engaging in the course and getting off to a good start?
- Are students successfully using course materials to complete assignments and activities?
- Are struggling students spending a lot of time/energy on course material and still not getting it, or are they not putting in the time/energy?
- Is the assignment feedback provided on student work helpful/effective and used by students?
- How many students access optional resources? Is it worth my time to create/add these to the course?
- Which course materials do students access the most? The least? Are there patterns of engagement?
- When do students access course materials? How often? Do they have enough time to complete activities?
- Is the class workload too much? How much time are students spending on course activities?
- Is my course inclusive, and does it provide diverse learning opportunities that all students can access?
Learner Engagement Analytics Dashboard (LEAD)
The Learner Engagement Analytics Dashboard (LEAD) aggregates data from Canvas, Kaltura and Engage eText to provide instructors with a unique view of student behavior in a course. Through 3 visualizations with numerous filters, instructors can look for insights around questions they may have, such as: are students accessing course content? How can I identify struggling students? Are there patterns of student performance?
Canvas
Canvas analytics provide data about students’ behavior, activity and performance in a course. Page view data can be used as a good approximation to student activity and not an absolute metric. It can help you see if activity occurred, and to view trends from week to week.
-
Canvas - Using Learning Analytics in Canvas (UW KnowledgeBase doc); check out the tab on Page Views and Participation.
-
Canvas New Analytics Introduction, Tour and Insights (video, 8.50 mins) CCS eLearning, 2020
Kaltura
Kaltura Mediaspace is used to host and stream media for educational use. Kaltura Analytics can show you numerous analytics about student access and engagement, including: who is accessing videos and when they are watching; the number of plays and unique users, total minutes viewed, which parts of a video are viewed more frequently, and the average drop-off rates (when you lose viewers).
-
Using Learning Analytics in Kaltura (UW KnowledgeBase doc)
-
My Analytics (Kaltura Knowledge Center)
Evaluate Social Learning
Note: Social learning can occur anywhere, and there are many tools and platforms where it does occur, in addition to any synchronous conversations that happen virtually or in real space.
Potential questions for this approach:
- Who are the students who engage in online discussions frequently and with a variety of other students? Are they knowledge shepherds that bridge groups?
- Who are the students who engage less frequently and/or with few other students in the online discussions?
- What can be learned about student attitudes and learning from an analysis of the content of discussion posts?
- Do students feel included, and that they belong in the course or program? Do they feel comfortable speaking up or asking questions?
Learner Engagement Analytics Dashboard (LEAD)
The Learner Engagement Analytics Dashboard (LEAD) allows you to filter on discussion activity by filtering the Main Activity Type to Canvas Discussions. This view will show you how many students accessed discussions pages in Canvas; it will not show you how students engaged with each other, however.
Piazza
Viewing Student Participation Analytics in Piazza (2.17 mins video) | Instructors: View Class Statistics
-
Piazza is a student-driven question and answer/discussion tool that instructors can use with a Canvas course. It provides a wiki-like functionality that allows students to post questions; instructors and students can provide answers and comments. Analytics show instructors which students are engaging the most, asking good questions, providing correct answers as well as seeing patterns of overall student participation.
TopHat
TopHat provides insights for instructors when integrated into a course. Depending on questions asked and how it's used, it may provide insights for how students connect and engage with each other.
Feedback Fruits (College of Engineering pilot)
Some schools and colleges are piloting Feedback Fruits, a tool that allows students to provide constructive feedback and reflect about group activities and group member participation for larger group projects.
Improve Learning Materials & Tools
Note: Instructors can look at engagement patterns and access to learning materials and activities to identify resources that may be overlooked, or are being revisited frequently. Most tools have some sort of engagement/access analytics, which might be useful when considering course iteration.
Potential questions for this approach:
- Does the course design align with learning objectives?
- Are course teaching practices effective for learners?
- Do learning activities and assessments help learners achieve course outcomes?
- What patterns can be seen in formative assessments to show students’ progress toward learning outcomes?
- Which learning materials and activities are MOST helpful and which are LEAST helpful for a given module?
- Are course materials, activities, assessments and teaching practices inclusive for all learners? Does the course support diversity, equity, inclusion, belonging?
Canvas
Canvas analytics provide data about students’ behavior, activity and performance in a course. Page view data can be used as a good approximation to student activity and not an absolute metric. It can help you see if activity occurred, and to view trends from week to week.
Look at page views and participation; where are students participating with course materials? Are they overlooking or skipping some of the activities or materials? Do you have clear navigation and due dates? Is there too much content?
-
Canvas - Using Learning Analytics in Canvas (UW KnowledgeBase doc); check out the tab on Page Views and Participation.
-
Canvas New Analytics Introduction, Tour and Insights (video, 8.50 mins) CCS eLearning, 2020
Kaltura
Kaltura Mediaspace is used to host and stream media for educational use. Kaltura Analytics can show you numerous analytics about student access and engagement, including: who is accessing videos and when they are watching; the number of plays and unique users, total minutes viewed, which parts of a video are viewed more frequently, and the average drop-off rates (when you lose viewers).
-
Questions to consider - are students watching the whole video? Check out when many students stop watching - is there a confusing concept at that time, or might it be content they already know? Is the video too long? Could you chunk it and add some engagement activities to keep student actively learning?
Individualized Learning
Note: Some tools are designed to customize course content and activities for learners; for example Engage eText or other digital learning tools (eg publishers’ tools with homework). Other tools have features that you can use to create a customized experience.
Potential questions for this approach:
- Do students need practice with more basic skills and concepts? Would students benefit from supplemental or more advanced course materials?
- Can students choose their own learning path, or choose content based on their learning preferences, eg. can they choose to view a video or read an article instead?
- Can I provide continuous feedback to support independent learning?
- If students have more opportunities to practice, study and review, will they do better with learning outcomes?
- Will individualized learning provide more flexibility for diverse learners and increase equity, inclusion and belonging?
Canvas
Canvas - most for-credit courses use Canvas for some course activities; consider some of these features to customize for students.
-
Quizzes with multiple choice questions can be set up to provide automatic feedback on correct and incorrect answers; this formative assessment provides additional information for deeper learning. Students can be sent back to concepts they’ve not yet mastered, or provided with new resources.
-
Multiple attempts for quizzes allow students to learn more, or practice skills and knowledge that need more work. Combining multiple attempts with multiple choice questions banks allows the quiz to be different when the student re-takes the quiz. Instructors can see the score on each attempt; you can also set it up so students can see their scores.
-
Mastery paths in Canvas course modules is also an approach that can be explored. This feature requires the course to be fully set up ahead of time using specific requirements and release content rules.
- Offer additional resources in your course - these can be for students performing at all levels, whether they would benefit from deeper learning and/or extra credit, or could use supplemental resources and review for more challenging concepts.
Engage eTexts and Digital Learning Tools
Engage eTexts and Digital Learning Tools
-
Textbooks and digital learning tools often have homework assignments, problems and quizzes included in the online platform. These are often designed to have adaptive or individualized learning features.
-
Find out more about digital learning tools; here’s an example from Pearson (but UW-Madison can help you access many other tools.)
Predict Student Performance
Note: While we don't have a predictive tool, think about how your course is structured, key assignments and your own experiences. Where are there key points in their learning that seem to indicate they are on-track, performing at a high level, or needing more assistance?
Potential questions for this approach:
- How can we use data (for example, collected in a learning management system or other) to predict how students may perform in a course, a degree program, or set of courses?
- What ways can we leverage learning analytics data to support student learning, whether students are struggling or begin to struggle?
- What learning analytics approaches can be used to help students from dropping out of college, or their degree programs?
- Can we use predictive analytics to support equity, diversity, inclusion and belonging? How do we mitigate biases?
Canvas
Canvas analytics provide data about students’ behavior, activity and performance in a course. Page view data can be used as a good approximation to student activity and not an absolute metric. It can help you see if activity occurred, and to view trends from week to week. Can you use these views along with the gradebook to determine whether you have students who may benefit if you reach out to them with extra credit opportunities or additional resources?
-
Canvas - Using Learning Analytics in Canvas (UW KnowledgeBase doc); check out the tab on Page Views and Participation.
-
Canvas New Analytics Introduction, Tour and Insights (video, 8.50 mins) CCS eLearning, 2020
For students: Students can use “What-If Grades” in Canvas, to calculate their total grade by entering hypothetical grades for assignments. Only students can enter and view What-If scores.
Visualize Learning Activities
Note: Many tools provide visualizations; this category as defined in the LA Functional Taxonomy also includes performance or grades, in addition to just how/when/if a student accesses course materials and activities.
Potential questions for this approach:
- How much time are students’ successful peers spending interacting with different course materials?
- What time of the week/day are students engaging with course materials? Can this help inform instructors about when to send out timely messages?
- Are students viewing important course content, or viewing other course content less, and can this help instructors structure their courses differently?
Learner Engagement Analytics Dashboard (LEAD)
The Learner Engagement Analytics Dashboard (LEAD) aggregates data from Canvas, Kaltura and Engage eText to provide instructors with a unique view of student behavior in a course. Through 3 visualizations with numerous filters, instructors can look for insights around questions they may have, such as: are students accessing course content? How can I identify struggling students? Are there patterns of student performance?
Canvas
Canvas analytics provide data about students’ behavior, activity and performance in a course. Page view data can be used as a good approximation to student activity and not an absolute metric. It can help you see if activity occurred, and to view trends from week to week.
-
Canvas - Using Learning Analytics in Canvas (UW KnowledgeBase doc); check out the tab on Page Views and Participation.
-
Canvas New Analytics Introduction, Tour and Insights (video, 8.50 mins) CCS eLearning, 2020
Access UW-Madison examples and examples from other institutions on the Learning Analytics Functional Taxonomy
UNTABBED CONTENT BELOW
Access Learning Behavior
Note: Most tools capture user activity and might be useful to view to see whether students' are engaging with course content and activities.
-
Learner Engagement Analytics Dashboard (LEAD)
-
The Learner Engagement Analytics Dashboard (LEAD) aggregates data from Canvas, Kaltura and Engage eText to provide instructors with a unique view of student behavior in a course. Through 3 visualizations with numerous filters, instructors can look for insights around questions they may have, such as: are students accessing course content? How can I identify struggling students? Are there patterns of student performance?
-
-
Canvas
-
Canvas analytics provide data about students’ behavior, activity and performance in a course. Page view data can be used as a good approximation to student activity and not an absolute metric. It can help you see if activity occurred, and to view trends from week to week.
-
Canvas - Using Learning Analytics in Canvas (UW KnowledgeBase doc); check out the tab on Page Views and Participation.
-
Canvas New Analytics Introduction, Tour and Insights (video, 8.50 mins) CCS eLearning, 2020
-
-
-
Kaltura Analytics
-
Kaltura Mediaspace is used to host and stream media for educational use. Kaltura Analytics can show you numerous analytics about student access and engagement, including: who is accessing videos and when they are watching; the number of plays and unique users, total minutes viewed, which parts of a video are viewed more frequently, and the average drop-off rates (when you lose viewers).
-
Using Learning Analytics in Kaltura (UW KnowledgeBase doc)
-
My Analytics (Kaltura Knowledge Center)
-
-
Evaluate Social Learning
Note: Social learning can occur anywhere, and there are many tools and platforms where it does occur, in addition to any synchronous conversations that happen virtually or in real space.
-
Learner Engagement Analytics Dashboard (LEAD)
-
The Learner Engagement Analytics Dashboard (LEAD) allows you to filter on discussion activity by filtering the Main Activity Type to Canvas Discussions. This view will show you how many students accessed discussions pages in Canvas; it will not show you how students engaged with each other, however.
-
-
Piazza
-
Viewing Student Participation Analytics in Piazza (2.17 mins video) | Instructors: View Class Statistics
-
Piazza is a student-driven question and answer/discussion tool that instructors can use with a Canvas course. It provides a wiki-like functionality that allows students to post questions; instructors and students can provide answers and comments. Analytics show instructors which students are engaging the most, asking good questions, providing correct answers as well as seeing patterns of overall student participation.
-
-
-
TopHat
-
TopHat provides insights for instructors when integrated into a course. Depending on questions asked and how it's used, it may provide insights for how students connect and engage with each other.
-
-
Feedback Fruits (School of Engineering)
- Some schools and colleges are piloting Feedback Fruits, a tool that allows students to provide constructive feedback and reflect about group activities and group member participation for larger group projects.
Improve Learning Materials & Tools
Note: Instructors can look at engagement patterns and access to learning materials and activities to identify resources that may be overlooked, or are being revisited frequently. Most tools have some sort of engagement/access analytics, which might be useful when considering course iteration.
-
Canvas
-
-
Canvas analytics provide data about students’ behavior, activity and performance in a course. Page view data can be used as a good approximation to student activity and not an absolute metric. It can help you see if activity occurred, and to view trends from week to week.
-
Look at page views and participation; where are students participating with course materials? Are they overlooking or skipping some of the activities or materials? Do you have clear navigation and due dates? Is there too much content?
-
Canvas - Using Learning Analytics in Canvas (UW KnowledgeBase doc); check out the tab on Page Views and Participation.
-
Canvas New Analytics Introduction, Tour and Insights (video, 8.50 mins) CCS eLearning, 2020
-
-
-
Kaltura Analytics
-
-
Kaltura Mediaspace is used to host and stream media for educational use. Kaltura Analytics can show you numerous analytics about student access and engagement, including: who is accessing videos and when they are watching; the number of plays and unique users, total minutes viewed, which parts of a video are viewed more frequently, and the average drop-off rates (when you lose viewers).
-
Questions to consider - are students watching the whole video? Check out when many students stop watching - is there a confusing concept at that time, or might it be content they already know? Is the video too long? Could you chunk it and add some engagement activities to keep student actively learning?
-
Individualized Learning
Note: Some tools are designed to customize course content and activities for learners; for example Engage eText or other digital learning tools (eg publishers’ tools with homework). Other tools have features that you can use to create a customized experience.
-
Canvas - most for-credit courses use Canvas for some course activities; consider some of these features to customize for students.
-
Quizzes with multiple choice questions can be set up to provide automatic feedback on correct and incorrect answers; this formative assessment provides additional information for deeper learning. Students can be sent back to concepts they’ve not yet mastered, or provided with new resources.
-
Multiple attempts for quizzes allow students to learn more, or practice skills and knowledge that need more work. Combining multiple attempts with multiple choice questions banks allows the quiz to be different when the student re-takes the quiz. Instructors can see the score on each attempt; you can also set it up so students can see their scores.
-
Mastery paths in Canvas course modules is also an approach that can be explored. This feature requires the course to be fully set up ahead of time using specific requirements and release content rules.
- Offer additional resources in your course - these can be for students performing at all levels, whether they would benefit from deeper learning and/or extra credit, or could use supplemental resources and review for more challenging concepts.
-
-
Engage eTexts and Digital Learning Tools
-
Textbooks and digital learning tools often have homework assignments, problems and quizzes included in the online platform. These are often designed to have adaptive or individualized learning features.
-
Find out more about digital learning tools; here’s an example from Pearson (but UW-Madison can help you access many other tools.)
-
Predict Student Performance
Note: While we don't have a predictive tool, think about how your course is structured, key assignments and your own experiences. Where are there key points in their learning that seem to indicate they are on-track, performing at a high level, or needing more assistance?
-
Canvas
-
Canvas analytics provide data about students’ behavior, activity and performance in a course. Page view data can be used as a good approximation to student activity and not an absolute metric. It can help you see if activity occurred, and to view trends from week to week. Can you use these views along with the gradebook to determine whether you have students who may benefit if you reach out to them with extra credit opportunities or additional resources?
-
Canvas - Using Learning Analytics in Canvas (UW KnowledgeBase doc); check out the tab on Page Views and Participation.
-
Canvas New Analytics Introduction, Tour and Insights (video, 8.50 mins) CCS eLearning, 2020
-
-
For students: Students can use “What-If Grades” in Canvas, to calculate their total grade by entering hypothetical grades for assignments. Only students can enter and view What-If scores.
Visualize Learning Activities
Note: Many tools provide visualizations; this category as defined in the LA Functional Taxonomy also includes performance or grades, in addition to just how/when/if a student accesses course materials and activities.
-
Learner Engagement Analytics Dashboard (LEAD)
-
The Learner Engagement Analytics Dashboard (LEAD) aggregates data from Canvas, Kaltura and Engage eText to provide instructors with a unique view of student behavior in a course. Through 3 visualizations with numerous filters, instructors can look for insights around questions they may have, such as: are students accessing course content? How can I identify struggling students? Are there patterns of student performance?
-
-
Canvas
-
Canvas analytics provide data about students’ behavior, activity and performance in a course. Page view data can be used as a good approximation to student activity and not an absolute metric. It can help you see if activity occurred, and to view trends from week to week.
-
Canvas - Using Learning Analytics in Canvas (UW KnowledgeBase doc); check out the tab on Page Views and Participation.
-
Canvas New Analytics Introduction, Tour and Insights (video, 8.50 mins) CCS eLearning, 2020
-
-
Access UW-Madison examples and examples from other institutions on the Learning Analytics Functional Taxonomy