New Quizzes Oops Error with Blueprint Sync
We use blueprints to manage the initial course content for the majority of our courses. We have migrated the majority of our Classic Quizzes to New Quizzes because we were originally told that we were going to be forced to use New Quizzes and we needed to start the process early to finish in time for the deadline.
In the past the New Quizzes copied into the associated courses without issue. We recently completed our bulk content load for the Spring semester and started receiving reports from instructors that the New Quizzes showed an oops error message that something went wrong with copying the content and there are buttons to Retry or Cancel. The blueprint syncs didn't report errors with New Quizzes failing to copy. However, after running a custom report using API calls, we found that about 40% of them failed to copy based on the assignment workflow state. If we use the "Retry" button it is able to copy the quiz, but it is no longer associated with the blueprint and gets removed from its placement in the Modules. Performing a re-sync also doesn't solve the problem.
We are still awaiting a response from Canvas Support, but we wanted to see if anyone else has experienced a similar issue with New Quizzes and performing bulk blueprint associations/syncs. If so, did you find any efficient ways to resolve the issue? We were given a suggestion for manually fixing the issues for one course, but it isn't scalable for the number of quizzes and courses that need to be fixed. We would prefer to avoid needing to reset the courses as some instructors started personalizing their courses for the upcoming term.
Hi James, We are exactly in the same situation and trying to resolve it by working Canvas support and our CSM. Please share with me if you know a work around or have come up with a solution since you've raised it a while ago. Thank you
We had to wait for Canvas Support to prioritize it as something for their engineers to fix. They fixed the majority of the issues, but didn't provide us with a way to fix it like they did. Based on the activity in the blueprints, it looks like they must have done something to allow the blueprint sync to fix the issue if it was still in an error state.
We had a few instructors later report issues with quizzes that were empty, but we suspect it may have been due to someone clicking the "Cancel" on the quiz error rather than "Fix" so it looked like it was fixed, but really the quiz was empty. For these we deleted the quiz that was missing questions and would re-copy the quiz from the blueprint and then add it to the appropriate module. This wasn't ideal, but worked well enough for the few remaining errors that were reported by instructors.
We are still waiting on more information about the cause of the issue and how we can ensure it doesn't happen again during our next content load. I'm wondering if there is some type of timeout/throttling issue occurring with New Quizzes that isn't reported to the Canvas API and is separate from the Canvas API Rate Limit. I have checks built into my script to monitor the Canvas API rate limit to avoid exceeding it and all the blueprint syncs that had errors with New Quizzes reported as completing successfully. Through some additional testing, I did find that even though the blueprint sync will report that it is finished successfully, the New Quizzes may still be in the process of being copied. Therefore, it seems they are running on different processes that are not fully communicating with each other to accurately report the status.
We wish the New Quizzes API was ready so that we could audit more to see if there are any other quizzes that don't have any questions in them so we can resolve the issue without having to wait for an instructor to report it. Using the Assignments API, we were able to find assignments where "is_quiz_lti_assignment" was true (meaning it was a New Quiz) and check the "workflow_state" to see if it was still in an error state ("failed"). However, this doesn't find the quizzes where someone already clicked "Cancel" and the quiz now looks to be fine, but doesn't have questions in it.
Thank you very much for the detailed response. Again, we are having very similar experience. We just heard back from Canvas Engineers that the issue has been resolved for FUTURE New Quizzes sync but for the existing New Quizzes that are not synced well or had errors like no content in them (just like quizzes with no questions in them in your case) will have to be fixed manually. We have about 100 associate courses or more cumulatively which we need to go in and make manual fixes for the multiple New Quizzes within those courses. We are discussing our options with Canvas team to work on the solution without any unwanted repercussions. They have provided us with two options so far: One to use course export tool and Second is the course reset option both of which are very manual processes and simply too time consuming.
I am new to the API world and have not used it. Having said that I understand the overall process you've undertaken to monitor the issue and would like to make a guess here that since NQ still operate a bit on the external tool side of the Canvas interface, the BP sync shows that the sync is complete when the New Quizzes '"shells" are created and then it lets the external app make copies inside the "shells" after the Sync process has finished from the BP course. i.e., the sync progress bar coming to a stop and the BP course registers that the sync is complete.
While this copy is happening, any changes made to the NQ such as editing it from the Modules page or hitting of "retry" or "cancel" button affects the quiz by breaking its link (probably treats the NQ as an edited Quiz) and prevents it from receiving further updates or finish the copy process.
I found that the while the NQ were getting synced, teachers of the associate course or admins had the opportunity to click on any of the NQ from Modules page. If they did that, (I tested this too) the Quiz will appear to have just the title and no content or questions in them. This opening/editing of a New Quiz from the Modules page also prevents it from getting the content copied across from the blueprints and somehow breaks the link. Which would then also prevent it from receiving further updates from future course syncs and gets listed in 'Exceptions'.
I am now thinking in the direction of how to fix the existing NQs that are in error state if Canvas does not come up with a time saving fix rather than manual individual associate course fixes. I'll update here if I come up with a better solution or if Canvas provides a good workaround.