Quiz Timer disconnection auto-submit

[willmoin]

Hi Everyone, 

Coming over from Blackboard I was really happy to see that the Quiz feature of Canvas had a decent Timer setup, that appears to keep counting down regardless if student leaves the test environment, and I had believed that this would then also 'automatically submit' on the students behalf in their absence. 

We're just putting together some guidance for our staff for setting online exams, and my colleague used a phrase that threw me a bit: "Note: If a timed quiz is automatically submitted after a student loses internet connectivity, the submission time stamp and time to complete the quiz may not align with the allotted time limit and/or the availability dates." It looks like it comes from this help guide What options can I set in a quiz?

So I didn't know what this meant, does it mean that the submission details are somehow incorrect on the Moderate page got the (Classic) quiz? Is this something our staff have to worry about? Would some students somehow end up getting more time than their peers?

I want our guidance to be as truthful and trustworthy and as transparent as they can be so I spoke to Instructure Tier 1 support to get clarification, and what the support contact says is that the quiz will auto-submit as per the timer, however in case of loss of internet there is disparity on the Moderate area where this keeps going until the student opens the quiz again. They suggested that the quiz gets submitted by the server but where it shows time taken will be incorrect. I asked about if a student never returns to a quiz, what happens, and this support guide was shared: How do I manually submit outstanding student quiz submissions?

In summary the support contact said: "all you need to know is that if the student disconnects the quiz will continue past the time allocated on the quiz, but the moment that they come back to the quiz it will auto submit and then will not be able to change anything on the quiz"

So it doesn't just auto-submit?

"in a way it does as once the time has gone pass the allocated time limit, the quiz will not be able to changed by the student again, and then the quiz will continue to show in progress and continue in the time showing for the students submissions until they open the quiz again and then the quiz will automatically close and stop the timer. This only happens when the student disconnects on the quiz in normal circumstances the quiz will auto submit"

OK, it doesn't work quite as I had envisaged, auto-submit online works when the student is present, but no biggy. However I then go on to test this with dummy students taking a 2-minute timed test, and even when the student leave the quiz page the auto-submit works flawlessly and the Moderate finalises after 10-20 seconds after the timer finishes changing from 'Time Up' to the actual timer limit, even though the student didn't return to the test. 

I queried it again, but this time I was told it was where students lose internet or leave the computer. Again on testing, no change it worked flawlessly even where the student never returns.

I couldn't get much further with this guy, he just congratulated me because it now showed that it was working properly, and said that the support articles above were aligned to this:

"When a student views a timed quiz and time expires, Canvas auto-submits the quiz. However, if a student navigates away from the quiz page, the quiz submission remains outstanding. On the Moderate Quiz page, Canvas generates a warning message about the outstanding submission. This lesson shows you how to submit any outstanding submissions for a quiz at one time." https://community.canvaslms.com/t5/Instructor-Guide/How-do-I-manually-submit-outstanding-student-qui... 

"Note: If a timed quiz is automatically submitted after a student loses internet connectivity, the submission time stamp and time to complete the quiz may not align with the allotted time limit and/or the availability dates." https://community.canvaslms.com/t5/Instructor-Guide/What-options-can-I-set-in-a-quiz/ta-p/683 

I said that I felt like the wording in the articles seems to be that auto-submit does not work in any cases whilst the student is away, and it does not instill confidence when I could see it working, wanted to tell our staff that it works, but Instructure resources are contradictory.  

We're not idiots here. If it's been an occasional occurrence that in the past some users have reported issues, it's generally safe but their are isolated incidence depending on many factors, I could understand, and I can help shape our internal training and guidance accordingly. But this lack of transparency is maddening! I wanted to have a chat with someone about the background to any observed problems but the guy (who I know to be very friendly and effective on any other day) just wouldn't go any further, standing by the system performing as the guides explain. 

So what have other people seen? Is there a history to this I'm missing out on? I notice that the second quote above is copied ad verbum by many other university websites. The Search box here doesn't return many results from a quick search, but might have been the wrong section.

Best wishes to all, Will (Liverpool, UK)

[chriscas]

Hi @willmoin ,

This has been a frustrating one for us too.  Basically the "auto-submit" functionality is somehow triggered in the web browser, rather than being a back-end process.  Effectively what this means is that if a student is taking a timed quiz and loses their internet connection, the quiz will not be submitted until that student logs back in to Canvas again (whether that's in a few minutes or a few days).  When the quiz does finally auto-submit, it shows that time as the submission time, which can really confuse everyone, since it may appear that they worked on the quiz for days when it has a timer or availability of only a few hours.  In reality, they did not work on it any longer than peers.

I feel the auto-submit should be a back-end process that doesn't require user intervention at all.  If that can't happen for whatever reason, i feel the submit date/time should at ;east reflect the end of the timer whenever the submission is eventually triggered.  I've discussed this with Canvas support and our CSM, but it doesn't seem like any changes for forthcoming.

This is a big caution to using late policies in a course, because quiz submissions can easily be marked as late when they really weren't.

-Chris

[willmoin]

Interesting, so it definitely does not auto-submit for you? Like I said - I shut down the PC, disconnected it from the internet everything and is still submitted 'back-end'. The only thing I can think is that maybe my tests which were only two minutes long should have been longer to evidence is operating in this way. Put's me in a tricky situation because I want to tell staff that this works, but your and also the Canvas guidance suggests it does not. I was using Classic if that makes any difference. Thanks, Will

[chriscas]

It's definitely a terminology issue I think.  A timed out quiz attempt will submit, but it will apparently only do so when the student logs back into the system again.  If the student's internet connection crashes in the middle of a quiz, and they don't log in again for a week, the quiz would appear to be unsubmitted during that week.  When they actually log in again, the partially completed quiz will submit itself and will record the date/time when that action happened (even if the quiz timer and/or available until time were a week ago).  This will cause the quiz submission to be marked as late, even though the student didn't actually do any of the work late.  It's definitely confusing, and is something I really hope Instructure revisits at some point!

-Chris