[ARCHIVED] Bug with Last Activity and the Zero Activity Report?

[mas]

I am seeing a potential bug concerning Last Activity and Zero Activity Report. Here is what I am seeing:

In the “People” section of course id 529, student Jacob Davis last visited the course on May 17 at 3:15 PM.

His total activity in that course is blank (should show at least 1 second of activity, correct?).

If we look at the “Page Views” section of his profile, course 529 does not appear anywhere.

When we run the zero activity report, the student does not appear in the report for this course. It as if he has attended the course, even though the "Page Views" section of his profile says otherwise.

My assumption is that the zero activity report is using the Last Activity date in the People section of the course, which is not accurate, and possibly a bug.

Please advise.

[James]

mas 

The last activity can show data even when the total activity time does not. I have had people register activity in a course, even though the admin page views do not show anything. On the other hand, the total activity is supposed to register interactions longer than 2 minutes, but given the way the documentation works, it doesn't say anything about interactions that are less than 2 minutes so they may or may not get counted.

What you're seeing could easily be explained by the student doing something, but spending less than 2 minutes in the course doing it.

I download the last_activity_at and total_activity_time each night for our early alert system. Here's an example from a student of mine who was mostly absent from my class after the COVID-19 shutdown. Notice that the last_activity_at changed on 4/29 but the total_activity_time did not.

Measuring activity in a course is unreliable at best and people often take it to mean something that it doesn't.

More information what is considered activity is is contained in How do I use the People page in a course as an instructor?  

Last Activity [6]—displays the date and time of a user's most recent interaction in the course. This time stamp updates when a user navigates through the course using the Course Navigation menu, replies to a discussion or submits an assignment or quiz, and when they access course files and pages. Please note that Last Activity does not include mobile interactions or group participation.
Total Activity [7]—allows you to see how long students interact within a course and counts page navigation only. Total activity time is displayed in hours:minutes:seconds. If a user has not yet reached an hour of activity, total activity time is displayed as minutes:seconds. Total Activity records any time spent viewing course content that exceeds two minutes. If the time between a new activity and the last completed activity is under ten minutes, all time between these two events will also be included. Total Activity does not include group activity or page views for videos that do not include intermediate page requests, such as a half-hour recorded lecture. To view specific page views for a student, view the student's course access report.

[mas]

Hello James, thank you. I was able to confirm that the student actually did have 4 page views in the course by clicking the "Access Report" on their profile. The Access Report showed 4 page views, all within 1 minute of each other.

The question that remains unanswered is why are those 4 page views not appearing in the "Page Views" section of the user profile? I chatted with a Canvas support agent who was able to view the student and course directly. Here is a direct quote from the agent:

"It doesn't look like there's anything out of the ordinary going on - it's most likely just because Page Views are generally pretty good but not always 100% accurate. So it looks like the page views missed a couple things for the student. Spotty internet can be one thing that causes the records to not match up."

I have no idea why the Access Report and Page Views would not align, even with spotty internet, as they should be pulling from the same database table. Regardless, this is not the first time that we have seen errors in Canvas data, which means we have to double-check every data point in every report we pull, adding significant amounts of overhead to our processes.

[James · Accepted solution]

The question that remains unanswered is why are those 4 page views not appearing in the "Page Views" section of the user profile?

To be clear, you are talking about the page views that are only available to admins? Page views are used in different ways and it can be confusing if we're talking about different things. 

There are page views that are HTTP requests. Everything comes into Canvas via one of these. That is what shows up for the admin, but there is some filtering that goes on there before Canvas shows it to us. Much of the CSS and JavaScript and miscellaneous stuff needed to display the page is not shown to us.

Then there are page views that happen when someone views content. There is an HTTP request (perhaps more than one) behind that, but Canvas consolidates that down to a single request.

One potential explanation depends on whether the student was using a mobile app or a web browser? The web browser is pretty good about getting things logged but there have been issues with mobile apps not getting logged properly. If I remember, some of the reason for the issue had to do with distinguishing an API call that was to view content from other API calls.

The information does not come from the same database. The page views (HTTP requests) is a massively huge table that is summarized for quick access by writing information to additional tables. It's that secondary table that the access report pulls its information from. It would be too slow and take too long to try and query the page views (requests) table every time someone needed an asset report.

[callie_pfister]

@James, can you answer a question for me about Last Accessed and the calendar?  Since students can access the calendar without opening a course, will clicking on Events on the course calendar register as course access and update the Last Accessed date (I guess assuming they looked at the calendar for at least two minutes, since that seems to be a limitation.)  I had assumed it would, but I just looked at an Access Report for one of my students, and I don't see any Calendar clicks listed on it.  (I guess these would be clicks for Events...?  I don't know if there is even an Event icon?) 

[James]

@callie_pfister 

Clicking on an event or an assignment from a calendar so that the dialog box pops up does not count as an access to the course. If the student clicks on the title link in the dialog box and it goes into the course, then it does count.

There is no need to look at something for 2 minutes to update the last activity. The 2 minutes is for the total activity column.

The update of the last activity does not necessarily mean that the person did something in the class though. I haven't checked in a while, but I have seen cases where the mobile app triggers activity in the class when the student is not even going into the course. I once had a student who had completed a course years before that was showing activity because the mobile app was doing some sort of scanning. In April 2020, I was having students show activity in the course that weren't doing anything in the course. I'm pretty sure she wasn't logging into the course, but she was doing stuff for her other courses.

[callie_pfister]

Thanks, James!  (And a P.S.:  I'm using your script to sort columns on the People page, and it is saving my life right now with various things our district is requiring.  I've shared it with my department, too, and we all agree that it is magical.  THANKS!!!)