Canvas Outcomes and New Quizzes Update

[jsailor]

For instructors that use outcomes and any form of outcomes-based grading or decision-making in their work with students, it is critical that all data is included in order to fully understand the competency levels achieved on course outcomes. As mentioned in Marissa’s Quarterly Update from New Quizzes - Q3 2022 post, our Outcomes team has been working toward this goal, ensuring data from outcomes-aligned New Quizzes data is included in the Learning Mastery Gradebook and the Admin-level Outcome Results Report. 

We are pleased to share that this work is on target and will be available in beta on October 27 and then deploy to production environments on November 9, 2022.

[BethCrook]

Great news! Thank you @jsailor !

[Hildi_Pardo]

This IS great!   

 Still hoping for media recorder to work in New Quizzes, and our district would be very happy (and many teachers will finally start using).

Thank you!

[vanzandt]

Hi @jsailor , I am told that when this is deployed, that the New Quizzes outcome will only report back for the institution level outcomes/reports, and not for sub-account level outcomes.  If this is true, that will not be sufficient for our program reporting needs. Outcomes are generally program/department specific and are loaded at the sub-account level at our institution. This will further delay our move to New Quizzes. Can you confirm whether this information/rumor is true or not?

[MayaBialik]

Will the New Quiz I just created and the students took last week show up or only those going forward?

[RandolphGonzale]

Good information on how to apply this.

[MrReed]

Did this feature go Live on November 9th?  My teachers are not seeing their New Quiz data in their Learning Mastery Gradebooks.

[jason_waltz]

I'm wondering whether this feature has actually gone live as anticipated. Just now, I tried the following:

• Created a New Quiz with outcomes aligned at the question level (did not appear in Learning Mastery Gradebook [LMG])
• Edited the quiz above to align an outcome to the quiz rather than just the questions (did not appear in LMG)
• Figured point 2 might have failed as a result of student 1 having already submitted the quiz. So, submitted the quiz as student 2 (did not appear in LMG)
• Created an additional New Quiz with outcome aligned to the quiz with no outcomes aligned at the individual question level. (did not appear in LMG)
• Searched Feature Settings for possible issues but found none.

So, my results lead me to conclude this feature has NOT been released as of yet. 

[amyslack]

We are seeing that this is working but not correctly. It appears like it is not fully populating, like students are missing, only showing 3 or 4 students for a class of 20 or not showing all the outcomes that were assigned to new quizzes. And this is even after more then 24 hours after the assessment was completed. Anyone else seeing this behavior?

[MrReed]

Now I have a teacher who is seeing new & different behavior.

She created the quiz & aligned outcomes in 1 course...and the data DID populate in the Learning Mastery gradebook.

She copied the quiz to another course. Here, the standard information will display in the "Outcomes Analysis" report, but does NOT show in the Learning Mastery gradebook.

I'm trying to provide guidance, but I can't seem to track down any new information on this feature or its implementation.

[jsailor]

@vanzandt, what you are hearing is correct given the way that New Quizzes are currently structured. That said, we know that outcomes and content are often created at the school/department level (sub-account). We are currently reviewing architectural changes that will need to be made to ensure use at the appropriate levels. We will be sure to provide updates as more information becomes available.

[jsailor]

@MayaBialik, the New Quiz that your students took should show up.

[jsailor]

@MrReed and @jason_waltz, we apologize for the confusion. It was brought to my attention that this feature was disabled for a short time as a bug was fixed. The feature is back on in production and working as intended, so you should no longer see any issues. @amyslack, I wonder if your issue was triggered by this as well. Please let us know if you are still seeing any issues.

[jsailor]

@MrReed, for your second post, do you know if the outcomes are institution level or course level? If course level, they will need to be added to the second course for results to show in the LMGB. If this is not the cause and the teacher is still seeing issues, we'll need to dig in deeper. The best way for us to do so is likely to have the teacher submit a support ticket with links to the two courses and quizzes. Our support team can then do an initial review and escalate to our engineering team if needed.

[jason_waltz]

@jsailor Thank you for the follow up. Outcomes for individual questions in New Quizzes are now populating within the LMG! Very nice feature!!

[Roosevelt1302]

ERROR: CAMBIO DE LA RESPUESTA EN LA EVALUACIÓN
Al momento de realizar mi evaluación en la plataforma de Canva de mi institución. La pagina lanzo un error, cerrando la sesion y pidiéndome iniciar de nuevo en la plataforma. Al momento de volver a iniciar sesión, me volvió al examen y cuando verifique de nuevo las respuestas que ya había marcado anteriormente, me di con la sorpresa que se habían guardado otras respuestas, diferentes a las que yo había marcado, después me volvió a lanzar otro error, cerrando por completo mi examen y lanzándome un código en la plataforma. ¿Me pueden explicar a qué se debe ese error o porque sucede?. No tan solo me sucedió una vez, sino que hace días, en otra evaluación, también la plataforma guardo otra respuesta diferente a la que yo había marcado. Son varias veces, y le sucede a varios alumnos.

ADEMÁS EN OTRA OPORTUNIDAD ME LANZO UN CODIGO DE ERROR: {"backup":false,"end_at":"2022-11-18T04:51:48Z","time_left":250,"hard_end_at":"2022-11-18T04:59:00Z","hard_time_left":249}

[michelle_coots]

Can we please have a toggle in the new quiz settings to indicate if the teacher wants the students' performance on the quiz to go into the LMG? There are times when a quiz uses items with aligned outcomes that are for practice and we don't want it in the grade - such as when the questions are reviewing basic concepts but not the full spectrum of proficiency levels.

This would be similar to how rubrics have the option to check a box specifying, "Don't post Outcomes results to Learning Mastery Gradebook". This could be a toggle on the new quiz settings page.

Thank you.

[michelle_coots]

When will outcomes import to a course with new quizzes? 

Currently, when we import a new quiz (or a whole module with new quizzes in it), the outcomes aligned to items do not get added to the course. These are root account-level outcomes on items in an institution-level item bank.

Even when the outcomes are separately imported to the course, the alignments do not reflect in the outcomes section of the course until after the quiz is administered. However, when an outcome is added to an assignment rubric it does show as an alignment even before students submit an attempt.

Thank you.

[amyslack]

Anyone else seeing the learning mastery gradebook just spinning and never loading. I have this happening for my whole institution and when I asked support they said a solution was coming but that was well over a week ago. This sort of worked for maybe a week but was only populating some of the students and only some of the outcomes and since then it has not populated at all. Its been way over two weeks. Anyone else seeing this?