Facilitators: Cid Freitag, Center for Teaching, Learning & Mentoring (CTLM) | Kari Jordahl, Learning Analytics Center of Excellence (LACE)

What questions do you have about the student experience? What data or information would help answer those questions, and are there strategies you can incorporate into your course to help you find answers? 

Slides  |  Recording

Presenter: Jonathan Gallimore, Psychology 

Jonathan shared his analysis and facilitated a conversation from considering questions in his course; for example, does attendance predict exam grades? Can students' access of course materials predict exam scores? How do late or missing assignments impact overall grades? 

Slides  |  Recording 

Participants from both years of the program discussed their action research projects, and shared some of the lessons learned and impacts. They spoke about the questions they explored, the data they accessed, and the actions they took or are planning to take.

• Liza Chang and Diana Frantz Anderson with CALS
• Sarah Ann Zurawski with School of Education/OT
• Christa Wille with Engineering (in collaboration with Kate Fu)
• Amanda Margolis with Pharmacy (in collaboration with Andrea Porter)
• Beth Janetski with Pharmacy (in collaboration with Casey Gallimore)

Slides | Recording

Presenters: Kari Jordahl, Dan Voeks & Kim Arnold; Learning Analytics Center of Excellence (LACE)

We have done numerous rounds of student engagement to find out what kinds of data might be useful for students. Find out more about what students have asked for, what types of approaches, tools and visualizations that students are interested in. We also discussed My Learning Analytics (MyLA), a student-facing tool that was created by the University of Michigan and that we're planning to pilot during fall semester.  

Slides | Recording

Presenters:  Cid Freitag, CTLM & Kari Jordahl, LACE

What resource, event or article has been impactful for you recently? We looked at a Big Ten Academic Alliance report about learning analytics efforts across numerous institutions, and briefly discussed several Unizin Summit sessions. Then we shifted towards thinking about AI and analytics, and looked at an example of a Tableau tool that uses AI.

Here's some interesting questions and resources to explore: 

• What are our peer institutions doing with learning analytics? UW-Madison is part of the Unizin Consortium and is also a member of the Big Ten Academic Alliance. 
  • Check out the Big Ten Academic Alliance Learning Analytics Interest Group Landscape Report to learn more about how LA is supported and organized, tools and approaches, programming, and opportunities and barriers. 
  • Did you attend the Unizin Summit? This year we heard about tools and approaches to support students, instructors and advisors, along with how AI is being incorporated in  teaching. Recordings are available on YouTube.
• Envisioning the future - AI is everywhere already- how do you see that intersecting more with learning analytics? Are you running into AI additions in the tools you use? 
  • The Instructure Community (Canvas) has an Artificial Intelligence in Education group with some interesting posts.
  • One tool example - Tableau is promoting their Einstein tool - to help ask queries. 
  • Check out this Kaltura blog from 2022 “AI in Education - All you Need to Know” which makes some good points, and is also not going to be all you need to know….
  • 2024 EDUCAUSE Horizon Action Plan: Unified Data Models - What is a unified data model for learning analytics, and why does it matter for higher education? What opportunities does it offer?  “Unified data models for learning analytics have been identified as one of the key technologies and practices anticipated to have a significant impact on the future of higher education. The 2024 Action Plan envisions a future.” 

Shared notes | recording

Presenters: Kim Arnold, Learning Analytics Center of Excellence (LACE) | Jeff Shokler, Office of Academic & Career Success (OACS) | Mary K. Thompson, Student Learning Assessment (SLA)

We discussed the high-level organization and structure of the DEEP framework, and heard updates about a couple of the working groups and their projects.

Slides | Recording

Presenter: Dariane Drake, Learning Analytics Center of Excellence (LACE)

Dariane talked about types of biases and helped us consider how we learn and reinforce them. We looked at case studies and examples, and reviewed tips for increasing awareness.

Slides | Recording 

Presenters: Diana M. McFarland & Janae Winston, Educational Psychology MS program in Learning Analytics

We heard from two students in the first cohort of the Educational Psychology MS program on learning analytics: Diana Marie McFarland and Janae Winston. They shared their capstone projects and reflected on their unique perspectives with learning analytics.

Slides | Recording

Group discussion 

We created a google doc and shared notes, insights and links and heard about the microgrant program, the Unizin Spring meeting, several NYU LEARN recent events about subversive LA and equity oriented LA.

Shared notes | Recording 

Presenters: Dan Voeks, Learning Analytics Center of Excellence (LACE) |  Mary K. Thompson, Student Learning Assessment (SLA)

Dan and Mary provided a brief review/introduction to our institutional data policy and data governance, and then we spent the rest of the session discussing and considering some real-life situations. For example: 

• You often work with student data and have access to student data, including grades as part of your role on campus. A close, trusted colleague wants access to the data. What are some things to consider? 
• You are a campus instructor and you want to engage students in the classroom by trying out a new software tool you found online. What are some things to consider?

Slides | Recording

Presenters: James McKay, Learn@UW  | Andrew Reinke, Learn@UW |  Keegan Shaw, Learn@UW 

James McKay showed us Piazza (for discussions); Canvas Analytics & New Analytics (which are no longer new but still branded that way); and TopHat (student response system).  

•  Piazza starts at 8:45 mins
•  Canvas Analytics/New Analytics - starts around 13.25 mins
•  TopHat starts at 24 mins

Andrew Reinke provided an overview about Kaltura Analytics, and showed us a couple of ways to access analytics which varies based on how you’ve embedded media into your course. He shared slides and a Kaltura terminology doc as additional resources.

• Kaltura Analytics starts at 27:56 min

Keegan Shaw discussed Atomic Assessments analytics, which integrates directly into Canvas and offers advanced quizzing functionality.

• Atomic Assessments - starts at 41 mins

Slides | Recording 

Presenter: Clare Huhn, Data, Academic Planning & Institutional Research (DAPIR)

Clare reviewed the overall structure of Tableau and how to access UW-Madison workbooks. These institutional resources can answer a variety of questions, for example: how are curricular subjects related to departments? What are the learning outcomes for a course? Are there differences in average grades between terms? Are there gaps in D/F/drop rates by demographic groups? 

You have access to the workbooks we explored with your UW-Madison credentials from either RADAR or the Tableau server web pages (you must use VPN if you are not on campus). Here's the workbooks we explored:

• • UW-Madison Academic Structure
  • UW-Madison Course Catalog 
  • Course Grade Distribution
  • Undergraduate Course D/F/Drop Rates
  • Courses Completed by Bachelor’s Degree Recipients
  • Course Demographics Profile

Slides | Recording

Presenters: Kari Jordahl, Learning Analytics Center of Excellence | Cid Freitag, Center for Teaching, Learning & Mentoring

How you design your course will affect what data you can see in various learning analytics tools. In this session we discussed tips for designing a course to help create useful data, including; 

• Review and reflect; what is your question? What tools/data are available?
• Identify a pivotal assignment or time in the course
• Use logical naming conventions and clear navigation
• Reminders about how to link to content (eg. Kaltura videos)

We also discussed using course blueprints or instructional design frameworks (Horton’s Absorb - Do - Connect) when designing a course, to ensure learning analytics is part of the overall course design and strategy.

Resources:

• Data Informed Design for Online Course Improvement (Chris Millet & Jessica Resig) book chapter in A Practitioner's Guide to Instructional Design in Higher Education.
• Transparency for Learning and Teaching Project: TILT Higher Ed

Slides | Recording 

Presenter: Cid Freitag, Center for Teaching, Learning & Mentoring

The Learner Engagement Analytics Dashboard (LEAD) can help identify students who may be struggling. Which visualization is most useful for you? Do you use other tools or methods to identify struggling students? 

Cid facilitated a discussion about how to translate data into action - using a framework that discussed Canvas tools and achievable actions - before the course starts, at the beginning of the term, during the term, and after the end of the course.  (Muljana, Pauline Salim, and Greg Placencia. "Learning analytics: Translating data into “just-in-time” interventions." Scholarship of Teaching and Learning, Innovative Pedagogy 1.1 (2018): 6.)

Slides| Recording

Presenters: Mary K. Thompson, Student Learning Assessment  | Saundy Solum, DoIT Academic Technology | Regina Lowery, Student Learning Assessment

Direct evidence of student learning (DESL) is an assessment approach that allows instructors to evaluate learning based on student performance on tasks directly connected to course learning outcomes. We learned how Canvas assignments can be linked to AEFIS Course Learning Outcomes, or CLOs and saw a demo of AEFIS. We also discussed the difference between grading and assessment, formative and summative assessment, the benefits of DESL, and how to get started using it. 

Slides | Recording 

Presenter: Cid Freitag, Center for Teaching, Learning & Mentoring (CTLM)

This session focused on one aspect of data literacy — being a consumer of data visualization products, such as those in the news, popular media, and data dashboards. By analyzing a visualization we focused on making meaning and thinking critically about the meaning and message it represents — preparing us to become a better consumers of visualizations. 

Slides | Recording 

Group discussion 

Spring conference season - what themes and new ideas were hot topics at events that you've attended (or that you wished you could attend)? Whether you had a chance to attend LAK'22 in March, the Unizin Summit in late April or the Indiana University LA Summit in mid-May (or other sessions we've missed) this conversation was open for all to share.

Shared notes | Recording 

Facilitators/presenters - team presentation facilitated by Kari Jordahl; Dan Voeks, Learning Analytics Center of Excellence | Shannon Harris, Learning Analytics Center of Excellence | Mary K. Thompson, Student Learning Assessment n

At this session we spent a little time providing information and updates on the LA landscape at UW-Madison, including the DEEP framework and our institutional data policy, and transparency around using data. We learned about enhancements for the Learner Engagement Analytics Dashboard (LEAD) and heard about Direct Evidence of Student Learning (DESL). We shared information about the new microgrant program - for exploring data-empowered educational practices/learning analytics in support of diversity, equity, inclusion and belonging. We also discussed the schedule and planned topics for fall LA CoP, while soliciting ideas for later sessions. 

Slides | Recording

Presenters: Chad Shorter, Learning Analytics Center of Excellence (LACE)  |  Maria Dahmen, Center for User Experience (CUE)

What types of learning analytics do students want? How do we empower students to use data about them, to support their own learning? 

We discussed what UW-Madison students think about learning analytics - what types of data and visualizations do they think are useful? How do they feel about data about them being used, when it’s used to support their learning? DoIT Academic Technology conducted several phases of student engagement over the past couple of years and this combined feedback from across those efforts.

Slides  |  Recording 

Presenter: Andrew Reinke, Learn@UW 

There are numerous ways to incorporate media into your course, and the analytics you can access will vary, based on your approach.  

• Individual media analytics
• Mediaspace My Analytics dashboard
• Canvas embedding options for best analytics
• Kaltura Gallery in Canvas Analytics
• Interactive video quizzes (for self-check/not for gradebook integration) 

Slides  |  Recording

Presenters: Cid Freitag, Center for Teaching, Learning & Mentoring (CTLM) | Kari Jordahl, Learning Analytics Center of Excellence (LACE)

We looked at a framework that can be used for learning analytics approaches. The Review-Amend-Apply framework (created at Indiana University) provides a useful and very approachable 3-step process. We discussed how it aligns (or doesn't align exactly) with the UW-Madison approach for learning analytics which focuses on a question that leads to data that leads to action.

• Review the information available to you about your students' behavior, your course, your teaching
• Amend the information you've gathered - are there other resources or sources of data? 
• Apply what you've learned - what modification or change might help? Consider evidence-based teaching practices. 

Slides | Recording

Presenters: James McKay, Shannon Harris, Cid Freitag, Kari Jordahl, all with DoIT Academic Technology

Sessions were offered frequently throughout the year.  

We explored the three visualizations in the Learner Engagement Analytics Dashboard (LEAD), and how to filter data to answer questions about courses, teaching, and students' activity in a  course. LEAD aggregates data from Canvas, Kaltura and Unizin Engage eText and offers unique insights about for-credit courses.

Slides (no recording)

Presenters: James McKay, Shannon Harris, Cid Freitag, Kari Jordahl, all with DoIT Academic Technology

Sessions were offered frequently throughout the year.  

We explored the three visualizations in the Learner Engagement Analytics Dashboard (LEAD), and how to filter data to answer questions about courses, teaching, and students' activity in a  course. LEAD aggregates data from Canvas, Kaltura and Unizin Engage eText and offers unique insights about for-credit courses.

Slides (no recording)

Presenter: Cid Freitag, CTLM

We focused on student engagement in a course, and identifying struggling students using LEAD visualizations. How can learning analytics help you identify students who may need more support? We also discussed actions you can take in Canvas throughout various times in the semester, to support students. 

(Muljana, Pauline Salim, and Greg Placencia. "Learning analytics: Translating data into “just-in-time” interventions." Scholarship of Teaching and Learning, Innovative Pedagogy 1.1 (2018): 6.)

Slides (no recording)

Presenters: Cid Freitag, Kari Jordahl, James McKay, Shannon Harris, all with DoIT Academic Technology

Do you use LEAD, Canvas New Analytics or Kaltura Analytics? Are you using Engage or other e-texts? We looked at some commonly asked questions about learning analytics and the Learner Engagement Analytics Dashboard (LEAD). 

Slides (no recording)

Presenters: Cid Freitag, Kari Jordahl, James McKay, Shannon Harris, all with DoIT Academic Technology

What worked well this semester? Are there changes you’d like to consider making in your course? Do you have instructional design strategies that worked well to leverage learning analytics? Interrogate your course data to discover which resources were most/least used, what times students accessed course content and activities. What strategies might help support teaching and learning?

Google doc (no recording)

Contributors: group discussion

Many spring conferences have learning analytics tracks or areas of focus, and one pandemic upside is that we can attend more conferences. This group discussion includesd updates from the Learning Analytics & Knowledge (LAK) conference, and the Unizin Summit 2021. 

Shared notes doc, links out to more resources (no recording)

Presenters: Kari Jordahl, LACE & Cid Freitag, CTLM

We facilitated a session at the Learning Design Community of Practice focusing on LA and course design. The learning analytics landscape and recent efforts at UW-Madison were discussed, and then we looked at analytics in tools that instructional designers typically have access to - eg. Canvas Analytics & New Analytics, Kaltura, etc. 

Slides  | (no recording)

Presenters: James McKay, Learn@UW; Kari Jordahl, LACE & Cid Freitag, CTLM

Have you explored learning analytics options available in Canvas? There are several locations in Canvas where you can access analytics, to answer questions you might have about your course and your students.

Slides | (no recording)

Presenters:  Kari Jordahl, Chad Shorter, Shannon Harris, all with Learning Analytics Center of Excellence (LACE)

This was a more general discussion around the use of learning analytics broadly. We discussed various ways to use learning analytics and talked about how course design impacts analytics, how analytics can provide insights and help you answer some questions, while potentially leading to more questions.

Slides  | Recording

Presenters: Andrew Reinke, Learn@UW & David Macasaet, L&S Learning Support Solutions 

We looked at several ways to incorporate media into a course with Andrew Reinke, and then looked at the visualizations and analytics that result from various methods.

David Macasaet discussed how he used Kaltura Analytics to review student engagement with video content in his course, and how student engagement improved after he made several modifications to the content.

Slides  | Recording