New Quizzes: Any plan to eliminate the 24 delay for Reports?

[EricBenshetler]

I set a deadline for students to complete each Classic Quiz before the beginning of the class at which we review the answers using the Quiz Statistics display.  Many students, of course, wait until the last minute.  Also, sometimes I give an in-class quiz where we immediately review how the full class did on the questions.

Waiting up to 24 hours to be able to view the reports for New Quizzes is unacceptable for these scenarios.

Is there a plan to eliminate the delay soon?

[greydon]

Hello @EricBenshetler 

Thank you for contacting the Canvas Community. We're sorry to hear you've been experiencing issues with the 24 hour delay for reports with New Quizzes. We understand you and others obviously want a faster outcome as you use these tools and features for grading purposes etc. Currently, there isn't an official date to expedite the process for the reports to be generated. We understand that there is a real need for a faster analytics report with New Quizzes, and currently it is part of the process for the Canvas Data 2 update. Ideally, we will have the reports coming within a 4 hour window although it is still being worked on by the developers. Canvas Data 2 has plans to shorten the latency of data, and we specifically outline improving analytics and creating quicker reporting for instructors within new quizzes here:

https://community.canvaslms.com/t5/The-Product-Blog/Further-updates-on-Canvas-Data-2/ba-p/496504 

[AndyWeyer]

From an instructor standpoint, this should be priority 1a for anything you're working on to update Canvas with. I am currently still waiting for my exam report to generate 46 hours after it was taken. We can sequence the human genome in just a few hours, and classic quizzes is able to provide data in realtime. I don't really see any excuse for why this has not been addressed yet.

If this is not addressed by the time the New Quizzes become mandatory in Canvas, I'll simply start providing my exams on paper and recommend to my institution that we should switch to a different LMS.

[EricBenshetler]

Thank you for the reply and the link to the CD2 blog entry.  Four hours would at least mean I could set a quiz deadline that's the same day that we'll review the answers.  The blog entry says the 4 hours is a "big improvement" compared to CD1 's 24 hours, which it certainly is, but it's still a big step back from the classic quiz functionality.  It still means that teachers will lose the current capability of giving and reviewing a quiz during the same class session.  

I hope that the development team gives improving this latency problem a high priority.

[bernhard_lohkam]

a) This is not a solution! This is a plan to maybe relief the pain a bit.

b) 4 hours delay is still unacceptable in my opinion. Considering that interactive learning is what most of us do or strive to do (incl. e.g. TBL etc). This is hampering progress in education and would be a reason for me to NOT consider Canvas in the future. Sadly we are stuck with it (at least for now). Considering available IT technology there isnt even a valid reason (to me) why this is taking that long. I could physically carry data faster and calculate the statistics in my head...

[amontguire]

Your link is from 2021 and there is STILL no improvement in lag time.  Why on Earth not?  Every other app under the sun, including YOUR own old quizzes can do this. Google, zip grade, every app on my phone, surveys on facebook.  Why does it take you so long to generate data?  It's  amazing that you marked this thread as "solved" when nothing has been solved.

[KarenPotter]

It is April 2023 and this continues to be a big issue at our school. When final grades were due Canvas has still not posted the analytics for a quiz done on the Friday. We had to stay very late to see the analytics. Why has this not been fixed the president of the company is asking me who is responsible? What can I say? The functionality of new quizzes for NGN questions is unparalleled compared to classic quizzes. I do regret making the switch though.

[KarenPotter]

I am also having significant delays of 2-3 days currently in New Quizzes Analytics, this issue needs to be solved.

[TamasBalogh]

Hi @KarenPotter and all,

I understand your frustration about the time needed to generate Item Analysis. My team is working on a new process which will significantly reduce the generation time and this work is our number one priority now. We aim to release the new, improved process in the upcoming months. 

[MayaCrosbyLA]

Hi - 

Thank you for this informaiton. Any update on the timeline for this work?

[amontguire]

Looks like it's fixed.  I now see a two analysis options under the reports tab, and the one on the left is very quick. Thanks Canvas.

[EricBenshetler]

Hi amontguire - Thanks for that comment.  Since my institution hasn't adopted the New Quizzes yet, I can't tell if this an acceptable solution.  Could you provide a little more information about the two analysis options under the Reports tab? For example, what is the wording of the "one of the left" and how quick is "very quick?"  If these look good, I'll gladly accept this change as a Solution so we can close out this issue.  Thanks!

[TamasBalogh · Accepted solution]

Hi @EricBenshetler, 
We implemented a new Quiz and Item Analysis report which replaces the old report. You can see my blog post about the release here: https://community.canvaslms.com/t5/The-Product-Blog/Faster-access-to-New-Quizzes-Item-Analysis-is-in...

"One on the left" means that for a limited period of time, we offer both reports for our users. You can see more information about this in the above-mentioned blog post.

The new process is on-demand and currently takes less than a minute to generate. The time depends on several factors, such as the number of submissions and items in a quiz, concurrent running generations, etc. 

You can see the differences in the comparison document here: https://community.canvaslms.com/t5/New-Quizzes-Resources/New-Quizzes-Item-Analysis-Report-Comparison...

Our guides are updated to reflect these changes and we created a comprehensive guide to explain our calculations and suggest how to interpret the metric values: https://community.canvaslms.com/t5/Canvas-Resource-Documents/New-Quizzes-Quiz-and-Item-Analysis/ta-p...

[EricBenshetler]

Looks good!  Thanks for the additional information and, more importantly, implementing the "generate report" on demand option for quiz results.  This puts to rest my concerns and now I'm looking forward to using the new quizzes.