cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
dspencer
Community Member

Quiz log interpretation

Jump to solution

Can you help me understand the attached screen shots of quiz logs.  I'm particularly interested in how the quiz can be "resumed" without a prior "stopped viewing" event.  Also it seems that there are instances when there is a answer submitted after at "stopped viewing" event but before a "resumed" event.

image.png

image.png

image.pngimage.pngimage.png

11 Solutions

Accepted Solutions
dspencer
Community Member

Thanks Chris.  That doesn't really help interpret the log.  It just shows me how to get it pulled up.  Thanks for trying though.

View solution in original post

tknapp
Community Contributor

Hi  @dspencer ​,

I did some testing in a sandbox course and here's what I found:

It seems like there are three times the "Stopped viewing the Canvas quiz-taking page" message occurs; when you're inactive within Canvas (after 30 seconds), when your mouse is inactive in a window outside of Canvas (after 15 seconds) and when you actually click outside of Canvas.

The "Resumed" message occurs when you move your mouse after being inactive or when you click back within Canvas after being active in another window.

I think the "Reviewed (and possibly read)" message is tied to scrolling through the questions and/or if your mouse hovers over a specific question.

Here's some more information from a prior questions Re: View Log Stopped viewing the Canvas quiz-taking window?

I hope this helps!

Tim

View solution in original post

tdelillo
Community Champion

I would add that the logs are somewhat inaccurate and not a solid picture of what the user is doing during the quiz. I did extensive testing when this feature first came out, even to the point of doing a screen cast of my quiz taking so I could compare the time stamp on the recording with the time stamps in the quiz logs. It was disappointing. I haven't seen improvements. I dissuade faculty from using it to try to "prove" anything regarding the student's quiz-taking behavior.

View solution in original post

dspencer
Community Member

I guess I'm not sure what the point is of this feature is then. If it isn't accurate and give me anything reliable what is its intended purpose?

If I have 20 pages of stops and starts does that not tell me the student is leaving the exam multiple times?

View solution in original post

lindsay_henry
Community Contributor

Thank you for breaking down the three possibilities as to why "Stopped viewing the Canvas quiz-taking page" message occurs.  There is a big difference between a student sitting idle (possibly pondering a question) and actually clicking a new browser (most likely cheating).  Is there any way for Canvas to differentiate between these two actions?  It's easy for students to cheat on online quizzes and this log can be a valuable tool in proving whether a student went to another page during a test.  I wouldn't want to accuse a student of cheating if they were just idle on the page!

View solution in original post

 @lindsay_henry , this is a good question and I agree it would be HUGELY beneficial!

Is there any way for Canvas to differentiate between these two actions? It's easy for students to cheat on online quizzes and this log can be a valuable tool in proving whether a student went to another page during a test. I wouldn't want to accuse a student of cheating if they were just idle on the page!

If they can tell the difference, they don't indicate it on the test log or anywhere else that Admin's (or teachers) would have easy access to.

I did a quick search and found this archived feature idea that sounded similar to what you are wanting - - my recommendation is to revive it as a new feature idea and add the extra information you'd also like to have included on the log.

View solution in original post

murphy_1527
Community Contributor

 @kona ‌ You mentioned we can use the log to see if students were working on the quiz and "got cut off." How do you know if this happened? What would the log entry appear as?

I had a student say that she got cut off from a test. She completed the test within in the window so that was not the issue. 

View solution in original post

My situation is different and I'm hoping someone can help me.  I'd love to view the log I've followed the directions and where the VIEW LOG should be - it doesn't exist!  I'd love to check up on a student.  HELP! Is there something I need to do in settings to make that appear?

View solution in original post

tl;dr Weirdness in Canvas Quiz Logs may appear because of various end-user activities or because Canvas prioritizes saving student's quiz data ahead of logging events. Also, there is a known issue with logging of "multiple answer" questions.

@astro-guy, I did take a look at the log you attached. Unfortunately, I can't definitively predict what happened on the users' end in that particular case, but I may be able to shed some light by explaining what happens behind the scenes.

'How exactly are the "stopped viewing the quiz log" and "resumed" sentinels generated...'

"Stopped viewing the Canvas quiz-taking page" is intended to represent if / when a student navigates away from the quiz, though it really just means that the user's browser focus was moved off the Quizzes page. For example, this will trigger if…

  • A student navigates to a new tab or window within their browser;
  • A student switches to a different application entirely;
  • A student's computer or browser pops-up an alert ;
  • A student is inactive on the page for a period of time (this can depend on the user's browser, so we can't say definitively "30 seconds" or whatever);
  • A student experiences an interruption in their internet connection.

"Resumed" is the opposite of the above -- i.e. a student's browser is re-focused on the quiz page.

'...and, specifically, what does it mean when the question is indicated as being "answered", between, and only between a stopped viewing and a resumed sentinel?'

Occasionally, a Canvas Quiz question may look like it was answered before it was shown or before activity resumed. This quirk can happen because Canvas prioritizes saving a student's answer and backing up that data before it sends a confirmation event to the Canvas server for Quiz Logs. This not only ensures that a student's answer is recorded above all else, it also helps prevent the students' quiz activity from being flagged by a user's ISP for sending too many network requests.

In the example you shared, you'll note there was a lot of student activity on that one question! Knowing that this question required the student to be typing paints the picture: I imagine that Canvas was busy recording the answer as the student typed, and thus delayed logging the "resumed" information until there was a sufficient pause in the answering activity.

Finally, I will note that there is a known issue with logging multiple answer questions in classic Quizzes. This is not a problem in new Quizzes.

 

 

View solution in original post

ml42
Community Member

Hello @astro-guy ,

I am somewhat relieved to hear that someone is experiencing a very similar situation as me. I wonder if the student of the quiz activity log you attached was using Firefox as they were taking the quiz? 

According to my instructor, I have never triggered any "Stopped viewing the quiz log" messages (or page blurs) on my quizzes using Chrome; however, I triggered page blurs on quizzes in a very similar order as your attachments using Firefox. Given that my quizzes are timed free-response questions, I will click into the textbox first thing in preparation of typing an answer, read the question, and then start typing my answer. In my small experimentation, I created a basic HTML page using what is likely to be the same HTML text editor used by Canvas (TinyMCE) and opened it on various Internet browsers. I found that Firefox treats clicking into the textbox as "leaving the page"; whereas Chrome and Microsoft Edge does not generate such a response. Although I cannot guarantee what I recreated is exact to Canvas' environment, I am eager to hear what thoughts or comments @jared has on this matter. 

@astro-guy, if you are able to share the browser the student was for the quiz, we may uncover an issue with the way the quiz log activities are recorded by Canvas. 

View solution in original post

astro-guy
Community Member

Very interesting ml42. The student was indeed using Firefox on an Lenovo Thinkpad running Windows 10.

View solution in original post

0 Kudos
28 Replies
chofer
Community Coach
Community Coach

Hi  @dspencer ​...

I'm not sure if this will help or not, but have you taken a look at this Canvas Guide?  How do I view a quiz log for a student?  It might give you some insights on your question.

dspencer
Community Member

Thanks Chris.  That doesn't really help interpret the log.  It just shows me how to get it pulled up.  Thanks for trying though.

View solution in original post

tknapp
Community Contributor

Hi  @dspencer ​,

I did some testing in a sandbox course and here's what I found:

It seems like there are three times the "Stopped viewing the Canvas quiz-taking page" message occurs; when you're inactive within Canvas (after 30 seconds), when your mouse is inactive in a window outside of Canvas (after 15 seconds) and when you actually click outside of Canvas.

The "Resumed" message occurs when you move your mouse after being inactive or when you click back within Canvas after being active in another window.

I think the "Reviewed (and possibly read)" message is tied to scrolling through the questions and/or if your mouse hovers over a specific question.

Here's some more information from a prior questions Re: View Log Stopped viewing the Canvas quiz-taking window?

I hope this helps!

Tim

View solution in original post

tdelillo
Community Champion

I would add that the logs are somewhat inaccurate and not a solid picture of what the user is doing during the quiz. I did extensive testing when this feature first came out, even to the point of doing a screen cast of my quiz taking so I could compare the time stamp on the recording with the time stamps in the quiz logs. It was disappointing. I haven't seen improvements. I dissuade faculty from using it to try to "prove" anything regarding the student's quiz-taking behavior.

View solution in original post

kona
Community Coach
Community Coach

We usually use it to get an "idea" of what might have been going on during the quiz taking, but as you stated, we definitely don't use this to "prove" anything!

dspencer
Community Member

I guess I'm not sure what the point is of this feature is then. If it isn't accurate and give me anything reliable what is its intended purpose?

If I have 20 pages of stops and starts does that not tell me the student is leaving the exam multiple times?

View solution in original post

kona
Community Coach
Community Coach

For us we use it as a way to gauge how honest a student is being. For example we'll have students who say they were diligently working on the quiz and got cut off. We can go to the quiz log and see if they were busy working or if they paused (for whatever reason) for a while before continuing. In some cases we have 5+ minutes between answering questions. In addition, we can also see if the student made it to the end of the quiz (viewed all questions) or if they really did get cut off at a certain point.

So no, not perfect, but yes, at least for us, still useful.

tdelillo
Community Champion

I agree with Kona, that it does have some usefulness to get a broad picture. It's just not as precise and reliable as it could be. To be fair, it is clearly marked at a beta feature. On the other hand, it's been "beta" for at least a year...

canvas_admin
Community Champion

While not answering your interpretation question, here's another use case of how it's been helpful to have this feature: A Canvas Thanksgiving