Does "New Quizzes" have anything to recommend it?

[StephanieAshley]

So, I just used New Quizzes for the first time and it seemed like a nightmare.  I found it extremely difficult and frustrating to use since it has so many parts and you can't navigate between parts without backing all the way out of the quiz.  The save feature is also inconsistent so it's also hard to know what is and isn't saving.  Then I found that it can't give you analytics for ?48? hours after the test?  It's been 60 hours and I'm still waiting.  Is there anything good about "New Quizzes?  After spending time in the forums, it seems to only do about half of what Classic Quizzes did and then it does it worse.  What am I missing?  What does it have to recommend it?

[Chris_Hofer · Accepted solution]

Hi there, @StephanieAshley ...

You are correct that New Quizzes is not a 1:1 comparison to current Classic Quizzes.  There are some new question types in New Quizzes (i.e. Hotspot) that are not in Classic Quizzes.  I don't know if you've seen this document, but if you go to this page: New Quizzes Transition Toolkit - Instructure Community (canvaslms.com), you'll find a link for "Classic Quizzes/New Quizzes Comparison Document".  I don't know if this document will be a direct answer to your question, but at least it might give you an idea of what is possible with New Quizzes vs. Classic Quizzes.

Hope it helps in some way!

[StephanieAshley]

Yes! Thank you!  This answers my question.  From what I can tell, there is no good reason to use New Quizzes and I will avoid it for as long as possible.  It has so many fewer features than Classic and it is so much harder to use.  But most of all, the quiz analytics don't function in a way that is usable so you can't get feedback after the quiz is done and that is really crucial for some many teachers.

[Chris_Hofer]

@StephanieAshley ...

You're not going to be able to use current Classic Quizzes forever.  Eventually, you will need to use New Quizzes.  I would recommend that you have a conversation with your school's Canvas administrator(s) or someone from your Online Learning / eLearning / Distance Education department.  They would be able to tell you what their plan is to move towards New Quizzes.  You might also want to read this posting:

We’re Listening: New Quizzes Features, Classic Qui... - Instructure Community (canvaslms.com)

I would also direct your attention to the New Quizzes Roadmap...which you can find here:

New Quizzes Hub - Instructure Community (canvaslms.com)

Scroll down the screen a bit, and you'll be able to see what things are coming to New Quizzes.

As you may know, there is little to no work being done on current Classic Quizzes...and the focus is mainly on New Quizzes.  You mentioned something about analytics.  You'll notice in the Roadmap that during Q3/Q4 of 2023, they have the following:

"Instructors can now access Item Analysis within hours or less, with meaningful data to ensure quiz effectiveness."

I'm not sure if this is what you are looking for or not, but it sounds like there is an improvement coming later this year.

Finally, if you've not had a chance to view this tutorial video, you might want to check it out.

New Quizzes Overview (Instructors)

Personally, I've found that in some ways, New Quizzes is easier for me to use than current Classic Quizzes.  I agree, there are some areas that still need some work, but I'll keep watch in the monthly Canvas Release Notes to see what enhancements are added to the New Quizzes interface.

[StephanieAshley]

So, the final update!  My quiz results took 5 days to come back because something went wrong and the engineers had to do something.  I don't have any more details than that.  They did, however, say that quiz item analysis results update about every 24 hours.  So it depends when students take their test compared to when that update occurs to know what results show up.  If you expand the analysis for any specific multiple choice question, it will tell you how many students gave each answer.  You can total those numbers to figure out how many quiz results it is using.  So if you have two classes taking it on different days, that should give you an indication of whether it is drawing from both classes or not.

However, in looking at my quizzes again to type out these steps, I noticed that my results had updated again from the last time I looked in a way that doesn't make sense.  (My lowest score got higher?)  Anyway, I give up on it.  It also doesn't update if you keyed in an answer incorrectly and need to change that.  I'm just going forward with the understanding that this function simply does not work and will not use New Quizzes again for a test where I need to analyze the results.  

[soumyasbh]

New Quizzes are a nightmare. There is nothing good about them.

[StephanieAshley]

This is interesting, but I'm still not sold.  They say they are listening, but that is hard to believe when it has taken four years of constant complaining just to fix a major bug if it actually gets fixed by the end of the year.  And while they are trying to rebuild trust with an apology post, the way it was rolled out in such an incomplete state has left them with a lot to rebuild.  Especially considering how many hoops you have to jump through to even give feedback.  That's why I'm here - trying to boost my status so I'm "allowed" to leave feedback.  Which by all appearances is an exercise in futility anyway if it takes 4 + years to get a single bug fixed.  I watched the video also and it conflicts with what tech support has told me in that after you build the quiz, you have to exit the build and then go back to the details and hit save.  So I'm not even sure what to do with that.  It is a constant frustration that you can't navigate between the details and the build page also. 

[Chris_Hofer]

@StephanieAshley ...

Even though you may not see employees responding to every posting here in the Community, they do read our feedback here in the Community site.  The Community site is just one of several ways that Instructure (the folks that make Canvas) gather feedback from us...the end users.  Here is a document that provides some more detail on the subject (check out the section under "Discover"):

What is the feature development process for Instru... - Instructure Community (canvaslms.com)

---

I watched the video also and it conflicts with what tech support has told me in that after you build the quiz, you have to exit the build and then go back to the details and hit save.  So I'm not even sure what to do with that.  It is a constant frustration that you can't navigate between the details and the build page also.

---

I could be wrong, but you may be misunderstanding something about the screens for New Quizzes.  If I am interpreting this part of your post correctly, you are indicating that you need to click the "Save" button to save your New Quiz?  This is not entirely true.  Let me try to explain...  When you create a New Quiz in your course, the first screen you would see (before you click on the "Build" button) is the assignment details screen...which is detailed here:

How do I edit the assignment details of a New Quizzes quiz?

Only when you make changes to this particular screen do you need to hit the "Save" button (or the "Save & Publish" button if you've not published your quiz for your students yet).

However, after you've clicked on the "Build" button to design your quiz questions and set other properties for your New Quiz, there is no reason you need to exit out of the "Build" screen to click the "Save" button again.  When in the "Build" screen, if you click on the "Settings" tab, all the preferences you change there are auto-saved.  There isn't any "Save" button, and again, there's no need to exit out of the "Build" screen to click "Save".

How do I manage settings for a quiz in New Quizzes?

When adding questions to your New Quiz, each question type has a "Done" button at the bottom of it.  Once you press "Done" for whatever question you are creating, it is automatically saved as part of your New Quiz.  Again, no need to exit out of "Build" and hit "Save".  Just as an example, here is the Guide for creating a True/False question in New Quizzes.  Near the end of the Guide, there's a section on "Save Question" where it indicates to click on "Done".  This is similar for the other question types in New Quizzes.

How do I create a True or False question in New Quizzes?

I took a glance at the Guide called How do I view reports for a quiz in New Quizzes?.  You are absolutely right that it says reports could take up to 48 hours to generate.  Have you tried reaching out to Canvas Support to report that your reports are taking far longer than 48 hours to generate?  If not, I recommend doing that a.s.a.p.  Canvas Support would be able to take a deeper dive into this with you.  If you contact Support, I would be interested in learning what they had to say...so if you could provide an update in this thread after you speak with them, that would be great!  How do I get help with Canvas as an instructor?

I do understand that it can be tough and frustrating to wait for new features to be added to Canvas or things to be fixed.  I've been in the same boat with some parts of Canvas at times.  While I'm sure that all of us would like Canvas to fix and/or develop "all the things", that's just not reality.  Like most organizations, Instructure has a budget to work with, and so there are some things that might have to get tabled for a time that we think should be higher on the priority list.  I've submitted some ideas here in the Community that I think would be great in the core Canvas product, and I've waited many years to have them developed...if they've been developed at all.  Sometimes I get quite a bit of support from fellow Community members...other times, my ideas don't get much attention at all.  So...I get it.

I hope this additional information has been helpful to you and clarifies things more for you (sorry if things weren't clear in my previous replies).  Looking forward to continuing the conversation.

[StephanieAshley]

So, I guess I have used New Quizzes twice now.  The first time was for a homework assignment.  I created the quiz just like the video shows.  The next night, I edited some of the questions.  Then the next day, my students went to take the quiz, but they got the original version.  I opened up the quiz and went to "build" and it showed my new edited version.  So I called tech support and they told me that it was because I hadn't saved the quiz.  They said that after I build or edit the quiz, I have to go back to the details page and hit "Save" there in order for the students to see it.  That was just a homework assignment, but it did mess things up for my first period and they weren't able to complete the assignment. So I see how it is "supposed" to work, but I also know what it did to me and what tech support told me.

I decided to give it one more shot and used it for a real test this last week. When I couldn't get my analytics to prepare myself for what to review the next day, I called tech support and they said I just needed to wait "48 hours" without specifying from what.  The due date?  The first student completion?  The majority of the students?  The last student?  After 48 hours from the last student taking it who is going to take it, I still had no analytics.  Tech support said I needed to wait 48 hours from when all students had taken it.  But I have students who are homebound, etc. who will never take the test.  So I escalated the complaint and haven't heard back yet.  48 hours came Friday afternoon. I'm still waiting.

I understand not being able to develop all the new things, but I would consider this a bug.  Classic quizzes tabulated statistics instantaneously.  With my admittedly somewhat limited experience in programming, it seems like it would be harder to program it with this kind of delay than to make it instantaneous.  I genuinely am baffled by why and how they made it this way.  And there are literally hundreds of posts about this in the community with hundreds of comments and upvotes going back to early 2020.  With a number of administrators threatening to go to another LMS because of it.  And it is going to take 4 years to address it.  During which time, it has actually gotten worse.  To me, that is just unacceptable.  

[Chris_Hofer]

@StephanieAshley ...

That's interesting that tech support told you that you'd need to save your new quiz.  I've not heard that before, so maybe it is true and I didn't know about this?  My role has been a Canvas administrator, so I've not had many experiences actually teaching full-semester courses in Canvas where quizzing is used as part of the assessment process.

When I think of bugs in software...it's usually because I've encountered an error message of some kind or a link that doesn't work on a website I'm navigating around.  On the other hand, since Instructure has it in their documentation that it can take up to 48 hours for reports to generate, that's a "by design" feature.  But, as I mentioned earlier, if you're finding that it's well beyond 48 hours, then that's something to bring to the attention of Canvas Support.  Another thing I'll mention...  if you find that any of the documentation is unclear (such as exactly when then 48 hours begins), you can always suggest clarifications to the Documentation Team for any Guide here in the Community.  For example, let's say you didn't think the information in the light-blue box in this Guide provided enough clarification on the 48 hours:

How do I view reports for a quiz in New Quizzes?

At the bottom of that page (and really, any Guide in the Community), you'll find a blue "Leave Feedback" button.  Here, you can fill out a short form for the Documentation Team on how you might suggest making the Guide clearer for everyone.

Hope this helps in some way.

[StephanieAshley]

I've had a lot of issues with Canvas keeping information.  For example, I will often update grades or enter zeroes for incomplete assignments and then these updates are gone when I go back to my computer.  Now I do lots of refreshes and going in and out of my gradebook to try to make it "stick."  Tech support has never really had a good answer for this. It may be that a similar issue was affecting how my quiz didn't save and tech support was just guessing about how to fix it, but I've used that technique ever since and it is really really annoying. I even went back and checked that correspondence with tech support and that quiz to confirm I'm not remembering wrong.

[TrishaMeyer1]

Hi @StephanieAshley ,

Have you made sure your browser is up to date, and have you tried clearing your cache? Canvas is really finicky about having a totally up-to-date browser. I've found that when things are really buggy, checking my Chrome version is a good idea.  (I don't know how often other browsers release updates or how particular Canvas is for those.) And a bloated cache can wreak havoc on Canvas operations, too. Little weird things such as those you are describing could be the result of "crossed signals," so to speak. Cleaning up your browser could help.

[StephanieAshley]

So an update on my Quiz Item Analysis.  I gave the test Wednesday with some students finishing on Thursday and filed my ticket Friday.  The analysis showed up Monday afternoon (five days after the test) so they closed the ticket with no response or explanation of why it took so long.  When I looked at the analysis again on Monday night, the results had updated even though no additional students had taken the test on Monday.  So I reopened the ticket to try to get some answers about 1) Why it took so long and should I expect this normally? and 2) How is it updating when more students finish?  How do I know how much data is included in each updated and how do I know when it's "done".  They sent me another email today that they wanted to close my ticket with no response or information about what is going on.  But I complained again and they kept it open.  We'll see, but I'm not optimistic.  Tech support seems to be decent at helping you fix immediate problems, but understanding the problems and helping you and/or the product improve going forward does not seem to be in their repertoire.

[StephanieAshley]

Part of the issue is that in a situation where some students take a test one day and the rest of the students take the test the next day (a really common scenario for schools on block schedule), no one even knows how that is supposed to work.  When do results show up?  How do you know if both classes are included?  My suspicion is that people who actually use and care about test statistics don't use New Quizzes because this feature truly isn't usable even if it works "correctly" in New Quizzes.  So no one has asked how it is supposed to work.  You already know enough to know that it functionally doesn't even without the details.

[paul_fynn]

New Quizzes still doesn't have feature. parity with Classic Quizzes, and doesn't report participation for quiz attempts back to Canvas New Analytics. It also isn't fully accessible.

As a development, it isn't covering Instructure in glory .......