Accuracy of Canvas Activity Log

[unsure101]

Hello, Could someone reputable confirm the accuracy of the canvas logs in terms of activy, timing and all activity that canvas reports. I need to understand how accurate are these logs. I was also informed that canvas uses AWS servers but was told by someone at AWS that they have problems and innacuracies all the time. But from a canvas perspective I need confirmation that these logs are not accurate and cannot be submissible to accuse a user for cheating or accessablity.

[chriscas · Accepted solution]

Hi @unsure101,

Honestly, this is one of the trickiest things to answer and talk to faculty at my university about.  I'd stress up front that things are really going to be different for almost each and every investigation of cheating, access, etc.  Since this is a public forum, I wouldn't suggest divulging any confidential info, but could you perhaps give us a little bit about the scenario you have to investigate right now?

In a very broad sense, the information you'll find in Canvas logs/access reports/etc is going to be valid, but the caveat is that you can never know for sure if the dataset is complete.  What I mean is that you won't necessarily be able to tell if a users internet connection dropped in the middle of a quiz, for example.  Canvas would just show the last time they did something in the quiz, and that it perhaps eventually auto-submitted at some later point.  The data about what the student answered at what time is going to be accurate, but it doesn't paint a complete picture of the student's experience.  In the past, activity done in the Canvas app wasn't always reported either, though I believe that was addressed maybe 4 or 5 years ago.

Canvas guides about quiz logs specifically call out that the logs shouldn't be used to investigate academic integrity issues, probably because of the issue mentioned above.

If you're in charge of investigating, you'll probably need to gather data not only from Canvas, but perhaps your login system or other sources as well.  I always preface anything I say in these situations with "according to the data I can see..." and "in my personal opinion, this seems to show...".  I will really never say anything with 100% certainty.  We have an academic integrity board which makes final decisions on issues, so they take evidence from the student(s), the teacher(s), sometimes myself, and anyone else involved, then make a determination based on all of the info.

Hope this helps a bit, even though it's probably not as concrete of an answer as you were hoping to get.

-Chris

[unsure101]

Spoiler

Thank Chris,

the basis for this issue is that the computer was offline or had a series of several connectivity issues that on the student end was showing they were offline. Also, when opening canvas the pages were blank and loaded blank were links and other information either was not showing or were not populating correctly. The webpage looked very skewed and some with only texts in a DOS type font. 



That being said, canvas however has tracked that the student has had several transactions during the time in which the student claims to had been offline or unable to reach the links and/or canvas webpages. 



The student claims they were unable to click or see anything but canvas logs are showing otherwise. Several pages of transactions actually.

[chriscas]

Hi @unsure101,

This is definitely challenging.  I believe the logs are reflecting that the server received a request and sent things back to the user (like a page, quiz question, etc), but they don't reflect that the user successfully received and displayed the page.  A bad piece of equipment anywhere between the user and the Canvas server could technically let the request go through but have the user see malformed/blank pages if they never fully received the response.  You could see if any of the transactions were the user submitting something, which could indicate things were working better than what the student says, but it's going to be really hard to definitively say what they ere really seeing unless they have screenshots at the time (and I'd scrutinize them if they were provided, we have seen students provide altered screenshots before).

-Chris

[unsure101]

@chriscas thank you. 

And if the student cannot provide screenshots. As they have stated they did not think to take these while having the claimed malfunctions. Is this the only course of action the student could provide to validate their claims?

[chriscas]

@hi @unsure101,

I'm not aware of anything else a student could really provide as proof.  Someone else in the community here may have some ideas though.  One thing you could also consider (again, this is really all up to the each school/college or even faculty member) is when the student informed someone about the issue.  Did they call in right away (if 24/7 is available), send an email immediately after their connection was resolved, etc.   This is not absolutely hard evidence at all, but if the student waited for a day to let someone know of the issue, it may be looked at differently than if they called a helpdesk within 5 or 10 minutes of experiencing the problem.

I will say that in cases like this, our faculty would generally give the student the benefit of the doubt, unless there is some sort of documented history of similar issues in the past for the student.  Again, I'll keep stressing that this is just what would generally happen at my institution, but things may be completely the opposite at other places.

-Chris

[unsure101]

Thanks @chriscas @I appreciate it.

[unsure101]

@chriscas