Parsing Outcome Results - PowerBI or...?

[TrevorCrandall]

Hello friends!

I am reaching out to better understand what the wider community, or those willing to have a quick email chat, is doing with regards to understanding their Outcomes.

Previously, we had a person from our Data & Research Team who would create a PowerBI App reflecting how our students performed on our outcomes, importantly mixed with demographic data such as race and gender, so our curriculum designers can understand the wholistic picture beyond grades. Some of the basic questions we ask of this data, are below.

Note - We have briefly looked at Intelligent Insights and our Teacher Preparation team isn't currently convinced the AI portion sitting on top of Canvas Data 2, would suit their needs.

• How did students perform on major assessments?
  • What rubric rows had the greatest level of student mastery?
  • Where are there opportunities for growth?
  • When students are not showing mastery, what are common pitfalls and challenges?
• Were there any differences in student performance based on their professor or by term?
• Are there trends in performance based on subgroups (e.g. race/ethnicity, gender, program, location)?

I was wondering if other schools did the same (PowerBI visualizations), or if they had a different piece of software supporting their needs? If you do use something else, would you be open to a brief conversation with my organization?

Much appreciated!

[TrevorCrandall]

Giving this post a nudge to see if anyone has thoughts post Instructurecon. Thanks!

[screviston]

I have been working on a similar task. I work in higher ed. We are starting to use outcomes to track success and challenges within a course. Sure we have always reviewed gradebooks for issues, but that only provides assignment level analysis. I have been using outcomes this year to aggregate results to the criteria level and outcome (competency) level to help understand larger trends and patterns. The below screenshot is a snapshot at Course Outcome level, each comprised of the connected criteria used across multiple assignments via rubrics.

I can drill into a specific Outcome to list success in each criteria, or report criteria by student. Anything further down you can typically just find in the gradebook. Course level reports are on demand, quick, and short lived via custom processing scripts. I can just run them anytime I want to review and get a current snapshot.

In addition I am also testing ways to aggregate course data up to the program level outcomes. The above example report is just a direct read of the mastery gradebook data. We do have some reports in PowerBI as well, which utilizes some demographic data. However those reports are more college level data such as our Core Ability outcomes that span all programs/courses, and only pulled once or twice per semester.

I wish there were more people interested in this, I see your comment is from summer and I am the first reply.

[TrevorCrandall]

Thanks for the awesome insight! Yeah our Power BI dashboard that uses the Outcomes as a baseline, but brings in demographic data, looks exactly like yours - with each course having their own drill down.

We are in the midst of a redesign and Power BI is the direction we are leaning in. We recently started accepting Canvas tables into Microsoft Fabric, so the plan is to make this a near real-time Dashboard that can be referenced any time instead of being generated once a term. Appreciate the insight here - I am guessing from the lack of tracking other school's may be doing the same - or relying on instructors and the learning mastery gradebook for their insights.

Thanks again!