Improve quiz analysis through robust reporting

[MPioRoda]

Theme Overview

Teachers need reports that are easy to manipulate, full of useful information, and efficient to access

What value could this provide to users?

• Quicker responses when accessing quiz and item analysis reports
  
• Item Analysis, Student Analysis, and Outcomes reporting would be easy to download and in a format that is easy to manipulate
  
• More information would be available in reports to help track progress or gaps

Why was this theme chosen to open for voting?

April 2023 • Quiz reports are key components to help customers understand outcomes and student performance. Reporting should be responsive, easy to manipulate, downloadable, and include the data necessary to help teachers and administrators make sound decisions on effectiveness and mastery.

Why was this theme chosen for prioritization?

June 2023 • While we will not be able to prioritize all ideas within this theme in the second half of the year, we will focus our efforts on improving the Item and Student Analyses.

[ProductPanda]

Note from Instructure

April 2023 • This theme was chosen to Open for Voting. 

How do Ideas and Themes work in the Instructure Community? 

How do I participate with themes? 

[nathanboettcher]

Canvas does already have some great quiz analysis, but if this featured more prominently in Canvas for faculty, that might increase usage. Further statistical analysis would be wonderful, but an easy summary of results should be readily available for faculty to become interested in the data enough to "drill down" to more detailed results.

[JamesSekcienski]

One of the highest priorities for this theme should be providing live data reporting in New Quizzes like there is with Classic Quizzes.  The quiz and item analysis reports shouldn't potentially take days to generate.

[LMacaulay]

The item analysis data in New Quizzes under Reports is useful, but needs to be able to be exported as a CSV or Excel spreadsheet file for easier analysis. Additionally, many faculty don't even realize the data that is on screen since there are so many clicks to even see the data.  

[CraigOgden]

I would also like to see item analysis of bank questions.  It would be nice to see all the questions from a bank and see which questions are missed the most and which questions seems to be the most difficult so we can remove them or move then to a more challenging bank. OR review those questions with our students more.

[lbach]

I would very much like to see an option to download all student responses in New Quizzes. Getting a summary statistics report is not sufficient for program review and assessment purposes, we need to be able to export the essay responses submitted by individual students.

[KathyPalm]

This is something our institution has been missing since the move to Canvas, being able to immediately view statistics on a New quiz after the quiz has been taken. We are very interested in this theme!

[ProductPanda]

Status changed to: Post Voting Review

[KristinL]

Status changed to: Prioritized

[jw2546]

Can you tell us, of the 20 feature ideas listed here, which led to the focus of "improving the Item and Student Analyses"?

[CaitG]

I agree this feature needs to be available.  We need to be able to easily track a student's progress over time.  Also, many of us conduct research on our educational interventions and I used to use the old quizzes as an easy way to do likert-scale questions and surveys. Now I have gotten myself in a pickle where i need to hand-transcribe 200 responses! The new canvas quizzes are so difficult to use, I think Im going to start using qualtrics.  Disappointing since the technology is clearly there. Please add this feature ASAP!

[TamasBalogh]

@jw2546 As stated, this time we are focusing on the ideas related to Item Analysis and Student Analysis:
- Faster Quiz and Item Analysis
- Y axis for the charts in both Quiz and Item analysis
- Explanatory descriptions for some metrics
- Answer Frequency summary to more question types
- Add student names to Answer Frequency Summaries
- Student Analysis report (in CSV format)
- Add time spent column to Student Analysis
- CSV downloadable for Quiz and Item Analysis

[JuliaCF]

Thanks for the update, Tamas! We have a faculty member who is wondering when the Student Analysis report (in CSV format) for new quizzes will be deployed - do you all have a time estimate for this? Thanks again!

[TamasBalogh]

Hey @JuliaCF,

Unfortunately, I don't have a timeline for this now, but I understand the Student Analysis is essential for you and a lot of customers so we are keeping it high on the priority list. 

[JThead]

I would like to see a column for duration included in the quiz reports, as well. This information is available under SpeedGrader, but is not captured in any of of the quiz statistics.