Canvas Outcomes & New Quizzes: Pt. 2

[BenFriedman]

In a previous blog, I detailed the 3 types of outcome alignments available in New Quizzes and some use cases for each. I also shared that our Outcomes development team is working on sending New Quiz data with outcome alignments to the Learning Mastery Gradebook and outcomes report exports. We’re very excited to bring this capability to Canvas as part of the transition to New Quizzes! 

In this post, we’re going to take a look at how outcomes-related scoring and reporting happen in New Quizzes. I’m excited to be joined by Chrystal Langston, a Senior Software Engineer on the Outcomes team as co-author to do that! 

We’re going to take a look at an example of how two students perform on an outcome-aligned New Quiz where each individual item carries an alignment. Once a student completes this quiz the results will be aggregated up across items with a percentage correct calculated that becomes an outcome assessment “score.” We generally do not suggest having only one item that assesses an outcome, particularly if it is multiple choice or a similar item type. 

Quiz Setup

Question	Outcome Item Alignment	Points
1	Outcome 2	1
2	Outcome 2	1
3	Outcome 3	1
4	Outcome 3	1
5	Outcome 3	1

Now let’s break this down to understand how a proficiency rating is derived from the quiz’s score percentage. If the quiz’s score percentage is equal to a rating’s score percentage, the student will be awarded the aligned rating. If the quiz result’s score percentage is greater than a rating’s score percentage but less than the next rating’s score percentage, the student will be awarded the lower of the two ratings. For an easy way to remember the scoring, think of the Price is Right game show, the lower price wins! #onedollarbob. 

Outcome 2, and 3 Details (using the Canvas default proficiency rating levels)

Title	Rating	Score Percentage
Exceeds Mastery	4	100%  (i.e. 4/4)
Mastery	3	75-99% (i.e 3/4)
Near Mastery	2	50-74% (i.e. 2/4)
Below Mastery	1	25-49% (i.e. 1/4)
No Evidence	0	0-24%

Outcome Results 

Next, let’s look at the quiz performance for two example students -  Ally Adams and Steven Smith. Outlined in the below two tables are their results. You will be able to see in the table how a student’s performance on the quiz items leads to a proficiency rating.  

Steven Smith Quiz Results

Outcome	Points and Percentage	Proficiency Score
Outcome 2	1 out of 2 (50%)	Near Mastery
Outcome 3	3 out of 3 (100%)	Exceeds Mastery

Ally Adams Quiz Results

Question	Aligned Outcome	Points Possible
Outcome 2	0 out of 2 (0%)	No Evidence
Outcome 3	3 out of 3 (100%)	Exceeds Mastery

Outcome Analysis Reporting

Below are screenshots of how these results are represented through the Outcomes Analysis reports for the two students which currently communicate results by a single quiz. As mentioned above, New Quiz results will soon go to the Learning Mastery Gradebook and .csv reports (and/or combined with assignment results) to give an overall picture of student mastery of outcomes over multiple assessments, a common feature request. Stay tuned for when this will be available as we work towards a release date!

In the first screenshot, we see whether each student met mastery and how many points they received on the quiz questions aligned to each outcome.

In the Outcome detail screens, we can see the distribution of students across mastery levels and the average score of students compared to where mastery was set for each outcome.