Community

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
bsmith14
New Member

"Time Spent"

I've been tasked with designing a custom report on students "time spent" on their courses. I'm still very new to Canvas, and the API responsibility is held by someone else so I'm not able to do the requests myself (grrrr) - I have to pass them through him. I've been trying to do some research on this but the sheer volume of data is staggering and I'm not honestly sure where to start. I'm also not working on the latest data, but rather older data so that I can mess around. I've seen references to the "requests" table, but don't see that at all on the list of tables that I've been given access to, so I need to be sure that this is where I should to start.

I'm already aware that this is probably not the best report to begin with, and the results can be skewed by any number of things but that's the request.

Any suggestions or advice is gratefully accepted. 

Tags (3)
8 Replies
ericwerth
Community Coach
Community Coach

Hi  @bsmith14 and welcome to the Canvas Community!  It sounds like you have an interesting project ahead!  I am not an expert on APIs but thought I would share this with the Canvas Admins‌ and Canvas Developers‌ user groups in hopes of bringing your question to the attention of the great folks in these groups.  You might consider joining one or both of these groups if you have not already done so.  There is great information here.

All the best and good luck!

0 Kudos
jeremy_lykken
New Member

Hi Bronwen,

The process for accessing the data from the Canvas API can be a struggle for first-time users, because it requires working knowledge of HTTP, APIs, programming, and data science. Below is a list of general steps you will need to take to get your course user data.

  1. Go to User > Settings and generate a New Access Token (you may need to be an administrator to do this). You will need this API key for your authorization header when making the request in step 3.
  2. Familiarize yourself with the Canvas REST API documentation to learn more about the API in general as well as to locate which data endpoint you want to use for your request. From the situation you described above, you will probably want to look at Analytics.
  3. Choose a programming language such as Python, Curl, etc. to make the HTTP request to the Canvas API. I like using Python myself and recommend using the requests library to make the API call. If you have little or no programming experience, I also know a district which has had a lot of success using Google App scripts.

This is an overly simplified response, but I hope it gets you headed in the right direction!

bsmith14
New Member

I think my biggest issue right now is that I won't be given access to the API. I'm a lowly SQL developer so all I can do is work with the data that they give me Smiley Sad.

jsavage2
Community Contributor

Hi  @bsmith14 

The requests table is part of the Canvas Data Portal (table descriptions here: https://portal.inshosteddata.com/docs). Your institution may or may not have access to it, depending on their license/subscription.

From the API, the information is accessed on per-user basis through the "List User Page views" of the Users endpoint: https://canvas.instructure.com/doc/api/users.html#method.page_views.index. The result is a PageView object for each http request: https://canvas.instructure.com/doc/api/users.html#PageView

Be warned, though, that even small institutions have massive requests logs that can take days or weeks to download. I just grabbed page views for all our users (we don't license the data portal), and even though we're a very, very small program, there were 13.5M entries that took about 10 days to retrieve (API calls are limited to 100 results per request).

That said, what you've been asked for is probably impossible to produce. The time spent on some types of activity can be modelled (Canvas even produces an "interaction seconds" estimate for certain page views). But a large part of any course is serving static content.

James
Community Champion

 @bsmith14 ,

The requests table is what people try to use for this, but I would repeat what Canvas does in their Canvas Data portal documentation.

Pageview requests. Disclaimer: The data in the requests table is a 'best effort' attempt, and is not guaranteed to be complete or wholly accurate. This data is meant to be used for rollups and analysis in the aggregate, _not_ in isolation for auditing, or other high-stakes analysis involving examining single users or small samples. As this data is generated from the Canvas logs files, not a transactional database, there are many places along the way data can be lost and/or duplicated (though uncommon). Additionally, given the size of this data, our processes are often done on monthly cycles for many parts of the requests tables, so as errors occur they can only be rectified monthly.

I just mentioned that same thing the other day to someone who wanted to use the page views to determine whether a student was cheating because their mobile device was loading pages while they were in a room taking a test and their phone was supposedly locked up outside the room.

The time spent in their courses is available, but it's not inside Canvas Data. Some data is available through Canvas Data, some through the REST API, some in both places, and some in neither.

The total_activity_time in seconds is available through the Enrollments API and every night, I download the enrollments information for every course and extract the last_activity_at and the total_activity_time for each student in the course and use it to create my own warehouse of that information.

My playing around is that those figures can be widely off, though, as students may be sitting on a page while "multitasking" and so they show lots of time in a course while and others may go into a course and leave once they've done their business and their times are low (but perhaps more accurate). There have been issues in the past where mobile activity may not be included, but I don't know if that was completely resolved. I've also had places where the certain things that are done show up as course activity, even when it wasn't. That's part of why I collect both pieces of information and if they show an activity on a certain date, but there's only a 1 second change in the total time spent in the course, I don't consider that they actually did anything in the course.

Intuitively, one would think that time spent in course would be a good predictor of success in the course. I can look at the students who have the worst grades in my class and they have little time spent in Canvas. On the other side, some of my best students are also spending little time in Canvas because they get it in class or they are just good at the subject and don't need to spend much time in Canvas other than to get the instructions and then turn things in.

All this said, there are some people in the Community who have spent a lot of time digging into the requests table and trying to extract that information. Much information can be gleaned from what is already written rather than trying to repeat it all here.

  • The Problem with the Requests Table: Robert has done a lot work and generously shared it with the Community. His name will appear all over the place when it comes to many things with Canvas Data and development in general.
  • How Do I Determine Time Spent on Site. Robert provides some SQL code here and talks about the noise in the requests table.
  • Requests Table: Joni has been doing a lot of development with Canvas Data as well, presenting on it at the annual Canvas conference: InstructureCon. I believe that she and Robert were working on Live Events together. They asked me if I wanted in on the discussion, but I've had other duties that have been consuming my time recently.
  • Compelling uses for the Requests table? Daniel contributed to this thread and was a person I was thinking of when I wrote that some people have spent a lot of time digging into things. I remember his posts from a couple of years ago listing some of the problems in determining information from Canvas Data.
  • Can we have access to students' time spent on the course site? Here Robert, Joni, and I all chime in about the requests table being the wrong place for this. Joni even made a feature request to get that added to the enrollment_dim table, but it didn't garner enough interest and was archived.

Ideally, the person who does the API would collect this data. You could analyze it later, but it really should come from some place other than Canvas Data. The person who asked you to find this probably has a misconception of what Canvas Data is ... the name is a little misleading ... and thinks that all of the information in Canvas is in there. With previous LMS systems, we hosted it on-site and had full access to the database and so anything that was in the system was available; Canvas Data does not provide that level of access.

ericwerth
Community Coach
Community Coach

Hello again  @bsmith14   There has been some great discussion in this thread.  The reply by  @James  brought up an interesting point, and that is perhaps there is place for discussion as to what you all hope to achieve by measuring time spent in a class.  Maybe there are better ways to achieve your goal than trying to measure time spent on a class in Canvas.  I've experienced similar issues with James regarding putting to much trust that there is a significant correlation between data I see in Canvas versus the time a student actually spends working on a course.  It is just hard to know what a student is doing while logged in to Canvas or what they are doing class-wise when not logged in.  Just as an example, perhaps some measure of attendance/week is more appropriate.  I also have found the discussion about estimating course load here to be interesting, How Much Should We Assign? Estimating Out of Class Workload — Rice University Center for Teaching Ex....  This isn't to say that trying to get data on student time spent on a class from Canvas can't have value, but perhaps it won't provide exactly what you all are hoping. 

jcoehoorn
New Member

No web site is EVER able to provide this information. Web sites loaded into a browser do not commonly maintain active connections to the servers, such that you could load a page like this question, spend literally **hours** reading and studying the content to be sure you understand it, and the server only ever saw one instantaneous request.

Of course, there are ways around this. You could write javascript to ping a web service within the site to know how long a page was held in the browser... but stop for a moment and look at your browser's title bar. How many tabs do you have open right now? Does it mean anything if those pages are still actively sending back heatbeat information to the web service?


I've seen (and worked on) sites that attempt to track this kind of metric, and they are **NEVER** able to produce numbers that have any kind of real meaning. They can produce numbers that may *seem* accurate, and even hold up to certain light acceptance testing... but it never stands up to real scrutiny. I've worked in another LMS that claims to provide this information (via plug-in)... but when you dig into what it's doing you find the numbers don't have much basis in reality.

I'll even go as far as saying anything that claims to provide this information is straight up lying. No screen yet invented is capable of knowing whether the user is really paying attention to it.

The only way this is ever useful is tracking aggregate engagement over time. This can work for a population (NOT an individual user)... but when I've talked about this with admins at my own institution, they've always only been interested in the information relative to auditing individual users, rather than measuring population engagement, and I believe this is the common case.

bsmith14
New Member

Many thanks everyone,  @jjones9 ‌ especially for the direction to the Enrolments API -

The total_activity_time in seconds is available through the Enrollments API and every night, I download the enrollments information for every course and extract the last_activity_at and the total_activity_time for each student in the course and use it to create my own warehouse of that information.

I'll have to have a conversation with IT as I can't find a field for total_activity_time anywhere in what they've given me. I have a feeling that we'll have to take the same route, as they flatly refuse to even consider getting the Requests table. It's just too big!

 @jcoehoorn ‌ I agree with you on your points about inaccurate metrics from websites. Our hope is that we can get an estimate of how long learners are spending on their courseware and possibly identify which courseware is more popular/effective - the only true reflection at the end of the day is how a learner performs in their exams. If they are going to open a page and then spend the entire lesson snapchatting their friends, there isn't anything we can do about that and there isn't any way to track it either.

ericwerth‌ we have class registers so can identify when a learner is in class, but I think the shift to what they are learning in their classes is definitely coming. I don't know the best way for us to go about this either, but the data is in Canvas somewhere! We just have to identify it. I watched Joni's video on InstructureCon and it did clear up some other questions I had as well.

Cheers for now