Canvas New Quizzes - Reports and Data Export

[lmrozinske]

I hope to clearly explain the need to improve the available reports in New Quizzes, and request that all reports include a data export option. New Quizzes has two out of the three report options that are available in Classic Quizzes. The available reports are Quiz and Item Analysis and Outcomes Analysis, and the unavailable report is Student Analysis.

1) Regarding the missing Student Analysis report - The Student Analysis report is critical.  It is one thing to know a single item is performing poorly (data available in the Quiz and Item Analysis report) and another to be able to identify the specific students who are having difficulty with that item or a group of items.  

2) Regarding the available Quiz and Item Analysis report - The report has lost some functionality. It no longer permits the user to drilldown on a specific answer option and discover which students selected that option. More importantly though is that the name of the item is not displayed and the data cannot be exported. Item review is extremely difficult when a user cannot track or analyze the detailed item stats for general health or distractor performance. The data is there, it is just not available in a workable format.

3) Regarding the available Outcomes Analysis report - There are two concerns with this report.  First an export (including student id and section) is needed and secondly, this report and the Learning Mastery report are not linked (ironically users can export outcome data from the Learning Mastery report).  

I see so much hesitation from the Canvas community because New Quizzes does not give users the same data experience as Classic Quizzes, and as educators, that is what we need. Users who have made the transition are finding that we have to spend hours manually exporting data (via copy/paste or data entry into excel) to achieve the same understanding that we use to have at the push of a button.  

I am hoping to discover if these concerns will be addressed in the next year, specifically the data export options. If exports are not made available, it would be to my benefit to move our program’s 400+ items back into Canvas Classic Quizzes.

[Chris_Hofer]

Hello @lmrozinske ...

It almost sounds as though you'd want to submit these items as separate Feature Ideas here in the Community website.  That way, others here in the Community have a chance to evaluate your ideas to see if what you are requesting is something they would also like to see implemented in Canvas.  Make sure each of your thoughts above are in a separate Feature Idea.  These Guides will help in formulating your postings:

• How do idea conversations work in the Instructure Community?
• What is the feature development process for Instructure products?
• How do I create a new idea conversation in the Instructure Community?

Let Community members know if you have any questions about this...thanks!

[hesspe]

@lmrozinske Thank you for articulating this so clearly and thoroughly. One serious concern you did not raise (I don't think) is the fact that (in Canvas's words) "Reports may take up to 24 hours to generate"

The timeline that is here says that the following will be resolved by July 2023:

"Student Analysis report to support teachers"

"Outcomes report downloadable CSV for teacher manipulation"

"Analysis downloadable CSV for teacher manipulation"

These are only three of at least 17 features that Canvas is promising to address by July 2023 and given the pace we've seen thus far, I find it a bit hard to imagine that they will accomplish everything on the list in the promised timeframe, but I hope they do.  Also, the wording quoted verbatim above is foggy, and in particular that is the case for "Analysis downloadable CSV for teacher manipulation." Does that mean student analysis and item analysis or something else. Enquiring minds want to know.

[AndrewBedinger]

Totally misleading to mark an item as "solved" with a link to page that says we'll maybe see this fixed in July 2023. Maybe a better resolution tag would be "pending resolution-learn more here"

[ErikaSpray]

Agree 100%. This is not solved. The existing functionality (or lack thereof) is woeful and impedes effective teaching and learning.  It is so frustrating that all the data is sitting there 'under the hood' and yet we can't access it - it's our data! 

[seharpe]

This is clearly not a priority for Canvas, which is shocking (and frankly unacceptable) given their apparent dedication to support teaching and learning. This was a known issue when New Quizzes was rolled out (circa 2021?). We were told July 2023 for an intended resolution. Now, they are saying this is on the roadmap for "Q1 2025 And Beyond"! (https://community.canvaslms.com/t5/New-Quizzes-Hub/ct-p/new_quizzes_hub#:~:text=New%20Quizzes%20Prod...), which sounds like "no resolution in sight any time soon." While I appreciate that the forced sunsetting of Classic Quizzes is on hold (seemingly indefinitely), it feels like some of these issues that have been raised time and time (and time) again by community members are in limbo.