New Quizzes: Display warning message if backtracking is disabled and student submits blank answer

0 Likes
(11)

BACKGROUND: In Classic Quizzes, if backtracking is disabled, students will receive a warning if they try to submit a blank answer to a question. 

PROBLEM: In New Quizzes, when backtracking is disabled, students do not receive any warning when they try to submit a blank answer. This has resulted in many of my students accidentally submitting blank answers and subsequently missing points. This has often occurred in situations when Lockdown Browser is lagging.

SOLUTION: Display a warning message if student tries to submit a blank answer on "New Quizzes" exams in which backtracking is disabled and then require that the student actively confirms they want to submit a blank answer.

 

Could we get this functionality added so students are warned before submitting a blank answer? Hopefully getting this fixed could be a priority since it directly affects students. 

26 Comments
Stef_retired
Instructure Alumni
Instructure Alumni
Status changed to: Moderating

@RogerThat 

Thanks for sharing this. Before we move it forward for a broader conversation, could you confirm that you have already asked Canvas Support to investigate the behavior?

RogerThat
Community Member

Hi @Stef_retired!  Thanks for the quick response! This is what I've done so far:

  • I submitted a ticket through the "Submit a ticket, Get Help" link on Canvas. My institution's tech support then contacted with me, and noted that the problem seems to be related to New Quizzes.
  • I posted to the New Quizzes forum page here

Is there a different support channel that I should speak with first?

Stef_retired
Instructure Alumni
Instructure Alumni

@RogerThat 

Would you please confirm that your institution's tech support escalated your ticket to Canvas Support, get the case number from them, and post it here? We'd like to make sure our engineers have determined that this is intentional design rather than a bug—so if local support hasn't yet escalated it, please ask them to do that for you. Thanks!

RogerThat
Community Member

@Stef_retired Sure thing! I'll contact them and organize that. Thank you!

RogerThat
Community Member

@Stef_retired Here's the case number: 06730323 

Thank you so much for your quick reply and your assistance with this matter!

Stef_retired
Instructure Alumni
Instructure Alumni

@RogerThat 

We've researched the case. The response from Canvas Support is that this is not intended behavior; it is a bug and we have associated it with an engineering ticket (Jira CNVS-50423). We've made sure that your case is linked with the engineering ticket so that you will receive updates on the progress toward resolution.

As this is now in the hands of Canvas Support, we've archived the idea.

Thanks for working with us on this!

Stef_retired
Instructure Alumni
Instructure Alumni
Status changed to: Archived
 
RogerThat
Community Member

@Stef_retired Thank you so much for the update and for linking this case w/ the engineering team! What would be the best way for me to track the progress of this? (e.g. is there a support or forum page) This will help me determine if/when I need to re-do my tests in Classic Quizzes.

Stef_retired
Instructure Alumni
Instructure Alumni

@RogerThat 

Assuming your institution's tech support initially submitted the case on your behalf, they will automatically receive updates as they occur. You can ask them to copy you on the ticket so that you will receive updates as well—or, you could ask them to keep you informed. 🙂

Stef_retired
Instructure Alumni
Instructure Alumni
Status changed to: Open

@RogerThat 

Thanks for your extraordinary patience with this, and we apologize for all the back-and-forth between various places. I just learned that upon reviewing the ticket, our Product team determined that the described backtracking functionality is currently working as intended.

So we'll move this idea forward now for broader conversation.

Please accept our apologies!

RogerThat
Community Member

@Stef_retired 😮😲😱🤢😰😵😖😧🤔😣😓😞🤕😣  

Hmmm... I'm rather confounded by this development  ☹️ 

I think if this goes unresolved, it will be problematic for a lot of programs since it directly impacts the student experience for test taking.

Can I confirm whether the product team understands I'm referring to the specific scenario where students are not allowed to backtrack? Meaning, if the instructor disables backtracking, and the student accidentally clicks "submit" without including an answer, the student has no way to go back to answer that question. Can I confirm that the product team does not think it is necessary to warn students that they are submitting a blank answer?

Some of my students described situations where the testing lagged when they tried to submit one answer, and when they click again, they accidentally bypassed a question.

RogerThat
Community Member

EDIT 10/7: Updated with additional information

Case #06730323
@Stef_retired 

Canvas Support requested that I provide an explanation as to why implementing my suggested idea would be beneficial. First I'll clarify that I have this bug when the following two options are set:

  • One question at a time
  • Disable backtracking

Here are some reasons to provide a warning if students do not answer a question in a quiz:

  1. Moral Standpoint:
    • If students accidentally click “next” without answering a question, they should be given a chance to fix their mistake and provide an answer. I mean, why not? Am I trying to set them up for failure, or am I trying to make sure they have the best chance to demonstrate what they know?
  2. Psychology Standpoint:
    • Students are already stressed while taking an exam. Some are particularly stressed and distracted since they are taking an exam online while being watched by a camera. We are adding even more stress and distraction to their test taking experience by excluding a safeguard that prevents them from accidentally skipping questions. And we all know that excess stress leads to decreased performance.
  3. Technical Standpoint:
    • Sometimes an internet connection or computer performance can result in freezing/lagging. If students click "next" when taking a remote exam and the connection lags, they may click “next” a second time to progress to the next question, but the test will interpret this as skipping a question. This has happened to my students.
  4. Design Standpoint:
    • Classic Quizzes currently gives a warning if there are unanswered questions, both when backtracking is allowed and when backtracking is disabled. New Quizzes should therefore continue this basic functionality.
    • New Quizzes currently gives a warning for unanswered questions only when backtracking is allowed. Why give a warning when backtracking is allowed, but not give a warning when backtracking is disabled and students have no chance to fix the question? This doesn't make sense from a design perspective, so New Quizzes should give a warning for the sake of design consistency.
  5. Competition Standpoint:
    • My institution just recently transferred from another testing solution to Canvas for exams. The other testing solution gave instructors the option to warn students if they left a blank answer. Classic Quizzes warns about blank answers. New Quizzes warns about blank answers (when backtracking is allowed). Why not have New Quizzes warn about blank answers when backtracking is disabled, or at least make it an option?
  6. Grades Standpoint:
    • There's gonna be that one student that could have received one grade but instead got a lesser grade because they accidentally skipped a question. This situation is certainly possible, as I'm sure every teacher has had a student miss a better grade by a fraction of a point. Or worse, there's gonna be that one student that would have passed a class, but failed because they accidentally skipped a question. Neither students nor instructors should have to deal with this type of situation that could have otherwise been prevented.
  7. Liability Standpoint:
    • I can foresee a situation where a student will purposefully leave one or more questions blank because they did not know the answer. They will then be able to say that the testing software did not warn them, and they will be able to argue for a second chance or for more points. By not giving a warning for blank answers, we are leaving instructors and institutions open for liability.
  8. Commonsense Standpoint:
    • In all the time that I’ve worked with electronic tests, I have never taken or proctored an electronic exam that did not warn the test taker when they left a question unanswered. I am confused as to why this warning would be purposefully left out – especially if it is already present in other contexts (e.g. there is a warning when backtracking is allowed, but not when backtracking is disabled).

This is one of those design flaws that results in students missing points they may have otherwise earned, which results in complaints from students, which results in decreased student satisfaction, which results in extra work for instructors, which results in complexities that nobody needs to deal with. There are many idea requests out there for things that improve the quality of life for instructors - but this idea request (bug report) is all about protecting the student test taking experience and the integrity of the grading process. 

Instead of asking "Why should we give a warning for blank answers?", I think better questions to consider are the following:

  • Why NOT give a warning?
  • Why NOT include a safeguard that would give students the best chance to demonstrate what they know?
  • Why would we purposefully design a system in a way that sets students up to accidentally miss points?
swind
Community Participant

Adding my voice in support of warning students when they submit a blank question.

We received an automated response to our support case 06652800 stating that the "...Product & Engineering team has reviewed this issue and determined it is functioning correctly: No warning will show to students if they do not answer a question in a quiz."

The "functioning correctly" argument holds no weight against the thorough reasoning that @RogerThat outlined above.

I agree that the question needs to be reframed: @Stef_retired why has Canvas decided to NOT include a warning?

kimm19
Community Novice

"...Product & Engineering team has reviewed this issue and determined it is functioning correctly: No warning will show to students if they do not answer a question in a quiz."

If that's the case, could we ADD in at least confirmation pop up window asking the student "Do you confirm an unanswered question?". In that perspective, it's not a warning but simply a confirmation from the student that the question was left blank not because of a lagging internet connection, Canvas/exam software issue, or any other technical issue.

The confirmation allows the student to give a fair opportunity to answer a question, and confirm they are skipping the quiz/exam question. The student cannot later argue that they missed the question because of technical issues, but prove they CHOSE to skip the question.

It's not issue of functionality but rather a possible area of improvement for Canvas in the future.

**I don't see how adding in a confirmation window would hurt Canvas, the instructor or student.

If Canvas does not want it's IT department to be flooded and overwhelmed with tickets such as this in the future, I highly recommend they consider this function.

Stef_retired
Instructure Alumni
Instructure Alumni

@RogerThat @kimm19 @swind 

Thanks for adding your thoughts. Please know that our product team is still evaluating this and following the conversation, so keep the comments coming!

vanzandt
Community Champion

Thank you @RogerThat for taking so much of your time to write a thorough statement in support of your idea, when you shouldn't have to.  There should not need to be more comments.  This statement, that was well written by Roger, is all that is needed:

-----

Design Standpoint:

  • Classic Quizzes currently gives a warning if there are unanswered questions, both when backtracking is allowed and when backtracking is disabled. New Quizzes should therefore continue this basic functionality.
  • New Quizzes currently gives a warning for unanswered questions only when backtracking is allowed. Why give a warning when backtracking is allowed, but not give a warning when backtracking is disabled and students have no chance to fix the question? This doesn't make sense from a design perspective, so New Quizzes should give a warning for the sake of design consistency.

-----

New Quizzes should be at parity with Classic Quizzes with these basic operations, and the fact that it already exists when backtracking is allowed just shows that this is a bug or oversight. This shouldn't even need to be an idea.

RogerThat
Community Member

Hi @Stef_retired !

I received the following email from support@instructure.com :

"Your Canvas Support case 06730323 has been set to a 'solved' status."

However, nothing was mentioned about what the final resolution was - e.g., did Canvas Support recognize the issue reported in this thread as an actual bug, or is the issue still considered expected behavior.

I sent an email back to support@instructure.com, but I want to try to cover all my bases on this since this is a pretty significant issue from the perspective of my students. Thank you!

RogerThat
Community Member

@vanzandt Thank you so much for taking the time to reply in this thread and show support for the issue we're trying to address. And thank you for the encouragement, too! We just transitioned to Canvas tests, and I put quite a bit of time and effort into developing my quizzes using the New Quizzes platform. I can deal with bugs that make my work more difficult - however, I can't tolerate bugs that needlessly compromise the student test taking experience. It's just a shame that all these efforts are being torpedoed by a tiny bug that seems simple enough to recognize, but I'm having to struggle so much to get recognized.

Stef_retired
Instructure Alumni
Instructure Alumni

@RogerThat 

Please continue to follow up on the (now-closed) ticket to request details, and I'll follow up with the developers as well. 

RogerThat
Community Member

@Stef_retired Thank you for following up on this! I went ahead and emailed Instructure Support back to keep the thread open, but I have not heard back just yet.