Quiz and Item analysis needs to be faster

0 Kudos
(22)

The new quizzes feature needs to have faster responses for the quiz and item analysis tool.  In the classic version, it computes in live time which allows faculty to begin analyzing the quiz right away.  Having to wait 24 hours (and in my experience longer than 24 hours), it delays our ability to post grades for the students in a timely manner.  Additionally, during midterms and finals weeks we often don't have 24 hours to wait to finalize grades as they have to be reported right away.  

33 Comments
Stef_retired
Instructure Alumni
Instructure Alumni
Status changed to: Open
 
amyslack
Community Participant

I completely agree. I support my school of about 150 teachers and during midterms they were suppose to analyze the data due a PLC meeting following midterms and non of them could access this data. It was extremely frustrating and I felt bad I had pushed them to start using New Quizzes more. This among some other quirks make New Quizzes a hard sell for our teachers. 

sliston
New Member

I also agree with the above comments. I appreciate how the original version stated that it would take up to 24 hours, however, the analysis was done in real time. This meant I was able to see the data in a few minutes in order to evaluate the validity of the exam questions. I completely agree with amyslack with waiting multiple days in order to complete the analysis and see the results will be very frustrating and makes the new version a hard sell. How can this be resolved? What is Canvas's plan to incorporate the strengths of the previous versions in the "new & improved" version of Quizzes?

lhodges1
New Member

I love many of the features in New Quizzes, but my students are constantly complaining that it takes over 24 hours to get their grades.  This absolutely needs to be fixed.  Since I'm the only instructor in my area that uses New Quizzes, I'm sure this is going to hurt my end of course evaluations.

NatalieIbendahl
New Member

Agreed. The delayed test reports are hampering my ability to respond to students with test results in a timely manner. Without test question data it is difficult to address student concerns about individual questions. 

RebeccaMcElhose
New Member

We depend on the test results to be given in a timely manner.  A lot of times, our content is built on material that is already learned, we need to understand any deficiencies in a timely manner so that we are able to address any issues.

LisaR
New Member

The ability to view the Item analysis following an exam needs to be much faster for instructors to be able to review test information and examine questions for issues such as miskey or unclear content. Item Analysis enables us to determine quickly and accurately which questions may need review. Without it, we are forced to use speedgrader and go through each students tests separately to figure out on our own which questions may need review. Students provide negative feedback on evaluations regarding the time we take to return grades and there are times, as others have stated, when we simply do not have this time available such as at the end of the term. 

I am currently at 6 hours waiting on a 30 question exam taken by 20 students and don't expect to see anything for at least another 8-10 hours. Then it only comes though in sections. You have to keep checking back before you can see all questions.

It was not like this a year ago. It originally took about 2-3 hours when I first began using New Quizzes. It has gotten increasingly worse. I don't understand why the previous quiz engine could do it as well other LMS but New Quizzes can not. Even the ability to simply grade by question would be a help since I can't view an item analysis until the next day.

KCStrieker
New Member

I was very excited for the change to LMS we went to in the last year. Especially with all of the technology adaptions that had to occur for blended learning.  I loved my prior test analysis we had on another LMS system because I was able to get feedback right away and weed through the questions that caused no problems.  When I was in school I wanted grades as soon as possible. Especially in a very competitive program.  This has hindered me from wanting to wait for the analysis for my students.  They spend so much of their time preparing for the exam I do not want to penalize them because of my grading system.  I have been spending quite a bit of time in the speed grader reviewing all the exams of my students to see problem areas because I do not believe they should wait for their grades that long.  With this said, I believe it is taking away from adequate time I could be using to be productive in another area to foster educating my adult learners.  At this time I do like so much about Canvas, but this really is an area I wish that could be fixed (not only for the benefit of instructors, but also the students)

Sylvia_Ami
Community Contributor

There's a similar conversation about a faster turnaround time for the Quiz and Item Analysis data over here: New Quizzes Analytics submitted by @hrowsey1 . I posted in that one since it was newer. I'm hoping we can consolidate this feature idea so it gets more momentum. Don't you agree @MattHanes ?

GrayWarhog
New Member

This meant I was able to see the data in a few minutes in order to evaluate the validity of the exam questions. I completely agree with amyslack with waiting multiple days in order to complete the analysis and see the results will be very frustrating and makes the new version a hard sell.

 

DG Customer First

TammyLiepe
New Member

Can anyone from Canvas on the back end answer these concerns?

LisaR
New Member

Currently 23 hours waiting for an item analysis on a 20 question multiple choice quiz from yesterday morning and have absolutely nothing in the report section yet.

I agree that it would be nice to hear from Canvas on this issue. 

dkahle
Community Member

I use "Quiz Statistics" in Classic Quizzes every day as a formative assessment. I can quickly see what percent of my class understood the overall concept and which individual questions they understood or need more help on.

I tried doing the same thing in New Quizzes - Build - Reports - Quiz and Item Analysis. The quiz statistics were not available immediately. It took overnight until I could see the report. This defeats my purpose of giving them immediate feedback beyond just an answer but to know whether I need to reteach a concept before moving on.

I am sad that Classic Quizzes must retire. Please make the Quiz and Item Analysis immediate for New Quizzes.

JenniferHubbell
New Member

Hi, I'm using New Quizzes, and it's been frustrating that no Reports or Quiz & Item Analysis will generate immediately.  Right now it takes 24 hours to generate.  I like to give my students immediate feedback because I think that reinforces their learning the best.  The delay in creating Quiz & Item analysis reports prevents me from checking out results before posting -- just to see if I need to re-grade any question in particular.  So, I can either immediately release a potentially faulty quiz, or make students wait a day+ to see their scores.  It thought the beauty of an online platform like this was immediate feedback.  I don't need a fancy report with graphs and charts.  Just a really quick table of results would be helpful immediately.  Thank you for considering this.

c_carrillo1
Community Participant

Item Analysis needs to be live.  Instructors need immediate access to this information in order to grade students properly.

lwri
New Member

If I could get quiz results immediately I would be so happy.

c_waterbury
Community Member

Timely feedback is essential. Having to wait 24 hours to see item analysis is unacceptable. Also, does this mean quiz data is also 24 hours old? Before faculty can more to new quizzes, item analysis needs to be live. 

JesCaron
Community Member

Having worked with numerous faculty in the past who use formative quizzing as a tool to inform Just-in-Time teaching, I can say that a 24 hour delay would be a serious detriment to their workflow.  While this may be a singular use case example, there are negative impacts in other situations as well - from the instructor who is trying to accomplish work in a more immediate fashion to the student who is looking to target studying of materials for the next quiz or exam (they need timely feedback in order to do this).  This feels like a step back from the current functionality and not one would be expected from a "new" tool. Item Analysis needs to be much closer to real time (not delayed by 24 hours) in order to best serve the needs of users.

bernhard_lohkam
Community Participant

Absolute must. We need to be able to communicate the results interactively with students. Classic can do it and to be able to replace this New Quiz needs to provide instant feedback just as Classic. There is no reason to take this long... Every phone can do these calculations in seconds.

For completeness, there is another similar topic New Quiz Analytics . A shame that this (as other) important points get "diluted" by having multiple entries. Is there a way to merge these?

hwesp
Community Member

Adding my voice - this definitely needs to be faster. Delayed data means delayed feedback to students about commonly missed questions and corrective steps to their learning.  I'd love for it to be live but could even settle for 1-4 hours so at least it is the same day as I graded tests.