May 27, 2020, update (version 15):
Another week, another update. This time I installed a new linter to check for crud and subtle issues with the code. It turned up a lot of little things and at least one place where I was using an array when I thought I was using an object. Here are some of the changes for version 15
May 19, 2020, update (version 14):
I've done a major rewrite of the script.
This is a major rewrite and updating will show a lot of changes. There is a good chance that any local customization you have done to the script will be lost. Those using a non-instructure.com domain for Canvas may need to reset their // @include line on line 5 (see the custom URL section below).
Canvas provides an Access Report for each student, but you have to click on each student's name from the People page to get it. This document will show you how to install a button on the People page that will create a .CSV file of the Access Report data for all students enrolled in the course. It will then show some ways you can obtain useful information from that data using Microsoft Excel. Although much of this data could be obtained through Canvas Data, this script makes it available to the instructor in real-time.
For those power users who are impatient, here are the quick install steps.
If you run into problems, be sure to go back and read the instructions.
Note that the Tampermonkey extension for Safari requires payment. This is for the author of the extension that allows the script to run. I am not asking anything for the script itself.
Canvas will give you an Access Report for each student that tells you how many times a student has viewed or participated in a particular content item. To obtain the list, you go to the Course Roster by clicking the People navigation link, then click on a student's name. Once you do that, you will get an item on the right-side navigation bar that says "Access Report for student's name". When you click on that, you get something like this:
It gives you the type of content (as an icon), the name of the content, the number of times a student viewed the content, the number of times a student participated in the content, and the last time the student viewed (or participated) in the content. It is sorted by the time the content was last accessed so you can see what they have been working on most recently.
Unfortunately, this information is available for just one student at a time, so answering questions like "Who viewed the PowerPoint presentation I told them to read?" becomes difficult -- you need to go into each student individually to see that information.
I've written a User Script that solves that problem. It fetches a list of all of the students in the class and then obtains the Access Report for each one, compiling all of that data into a single Comma Separated Values (.CSV) file that can be opened with a spreadsheet like Microsoft Excel.
This document shows you how to install that script and then analyze the data that you get from it.
Installing the User Script
The script automatically runs on any page that matches https://*.instructure.com/courses/*/users. This is the main People page if your site is hosted by Instructure without a custom instance. If you have a custom URL, like canvas.university.edu, then you will need to modify the script to get it to work.
The specific steps to do this vary depending on your browser add-on. In Greasemonkey, click on the Greasemonkey pull-down, choose Manage User Scripts, find the Access Report Data, right click and choose Edit. In Tampermonkey, click on the Tampermonkey Icon, choose Dashboard, and then click on Access Report Data.
In either case, you need to change // @include statement on line 5 to match your instance. In the case of canvas.university.edu, you should change it to https://canvas.university.edu/courses/*/users. The * is a wildcard that will match any course.
There are two configuration variables that can be set within the source code to alter the functionality of the script.
In the June 4, 2016, production release, Canvas consolidated the buttons that previously appeared on the People page under the administrative cog.
The script was updated on June 6, 2016, to reflect that change and move the Access Report Data into that same cog. Note that the demonstration videos have not been updated to reflect the change.
When you expand the menu by clicking on the cog, then you can choose the Access Report Data item.
Click on this and wait.
There is now a progress bar that appears once the list of students has been downloaded and it is fetching the access reports.
It has to make an API call for the list of students and then generate the Access Report for each student in the course. This only took 8 seconds for my class with 46 students and up to 112 content items, but we're only one month into the semester and it will probably slow down later. It took 12 seconds on a course with 81 active students and 126 content items. It took 15 seconds for a class with 35 students but 470 content items.
Early on, really large classes were having timeout issues in Chrome. Switching to Firefox helped. Now I wonder if the problems were related to the requests failing and so I hope the changes I made with version 14 have improved this so that it is no longer an issue. I've made some improvements to the code with version 14 and let Chrome run for 4.5 minutes in a class with 29,615 items without issues
If you have a really large class, you may need to tweak the settings. The call to get the enrollments is relatively expensive compared to the call to get the usage (access report data). In a class of 300 students, it was fine, but in a class of over 10,000 students, it was timing out. It is supposed to retry failed attempts, but there was some data loss that I haven't tracked down yet. It's best to avoid exhausting the x-rate-limit-remaining header. Increase the minTime or decrease the maxConcurrent values. Normally, the minTime is the more important one, but if your requests are taking a long time to process (like the enrollments were), then you may decrease the number of concurrent requests. Reducing maxConcurrent from 40 to 20 avoided the lost data issue in the 10k+ course and only added 4 seconds to the download time for the class with 300 students.
The course with 81 students actually had 233 students in it, but it was a resource course for faculty and only 81 had bothered to go into the course to do anything. That brings up an important note about the Access Report.
The Access Report only provides information on students who are doing something in the course.
Access Report data does not return information about who has not done something, but Excel has an option that will allow you to see this.
Another thing to note is that additional information is available through the Page View data, like the exact times when students did something and which browser they were using at the time. However, faculty don't generally have access to this information and you have to load the information for each student and sift through it to see which applies to the particular course. This takes way more than 8 seconds for little gain.
The Page View information is available through Canvas Data. However the data there is not current, running about a day or two behind real-time. It also has limited availability and requires additional resources to analyze. Page view data still occurs when a student does something. If they're not doing anything, there is nothing to record.
Finally, links to URLs that are contained within assignments or pages are not included here. The External URLs are just those that are linked to items in modules, not a link in a page full of instructions.
Okay, you've installed the script, clicked the button, and opened the file in Excel. Now what?
Well, you get a bunch of raw data that will need manipulated before you can tell anything useful. It looks pretty intimidating, you have a report that looks like this (and this is just the first 10 rows).
Note that the SIS Login and SIS User ID may not be there, depending on the permissions you have within Canvas.
Here's what that first entry looks like in JSON.
"display_name":"Discussion 4: Hypotheses and Errors",
"readable_name":"Discussion 4: Hypotheses and Errors",
What's being displayed in the HTML version of the report is contained are the asset_class_name, readable_name, view_score, participate_score, and last_access. The User's name is displayed at the top of the report, but the user_id (1278402) can be used to identify the student (in this case Cubic Dream) and the user script merges the user information and the access data together into a usable format for the user. By the way, all of the user data have been anonymized, but the assignments and access data are from my actual class. Cubic Dream isn't my student's real name and 1278402 isn't Cubic's real Canvas User ID.
In statistics, I emphasize the importance of having a good data dictionary. You need to know what the values represent.
Note that section information was added after the original script was released and are not included in the videos.
Some questions can be answered using just the raw data.
If you're going to examine the raw data, you will want to turn it into a table first. To do this, go to Home > Format as Table or Insert > Table. This allows you to filter or sort on a column, which will greatly increase your productivity later.
Let's say we wanted to know who participated (and how many times) in Discussion 3.
I purposely started this one off in a novice way, to show what people who may not be familiar with Excel can do. There are more efficient ways of doing this.
You'll want to sort by title to group the content together. You could use the Code to group, but it wouldn't be alphabetical.
I then used filters to select the external URLs. Those can be found under Category (external_urls) or Class (content_tag) and found the one I was looking for.
Here is a video walk-through of the three items mentioned here.
Sometimes you want a broader picture than just what a student did on a single assignment. To accomplish these, you'll need to create a Pivot Table.
To create a pivot table, you need to go to Insert > Pivot Table and click OK.
Once you're there, you will probably want to drag the Display Name or Sortable Name down to the Rows so you can break things down by students. Every one of the items in this section starts off the same way.
Pivot tables allow you to insert slicers to quickly filter the data or time slicers to view data over a particular time period (you must have a date/time field to do this, but we have first access and last access to pick from)
This whole Access Report project grew out of a desire to know how many times students had gone into the discussions and at least viewed them.
What I'd like to know is how many times did a student view or participate in a discussion this semester.
Here is a video that shows all of that in action.
The Course Roster (People) page shows you when the last activity of a student was and the total amount of time spent in the course. It does this in alphabetical order. Unfortunately, last activity could be just logging into the course, it doesn't mean they did anything else once they got there.
As a side note, if you have installed my Sort a Roster Canvancement, then you can click at the top of any column to sort by that column. You can find sort by section, by the time they last accessed the course, or even by the total amount of time spent in the course.
You can filter the information in the Access Report Data spreadsheet by time.
Remember that only students who have data for that time period will show up. There is no easy way to get a list of students who aren't doing a particular thing.
Here is a video that walks you through the report.
A quick table is a table that shows you information about a particular area. What we're going to do here is break down the information by the Class. You could just as easily choose the Category and some might find it more useful.
Here is a video that shows all of this in action.
As mentioned above, what we are looking at is the Access Report data and it doesn't include information about students who are not engaging in your course. Luckily, there is one checkbox in Excel that we can check to get that information. The student will have needed to do something, anything, so that their name is in the Access Report data, but then we can see what they have not done.
Here's a video showing how it works.
Tampermonkey should attempt to update this script automatically for you. In case you have turned off that functionality or would like to update sooner, you can click on the Tampermonkey icon and choose Check for userscritp updates.
The update process keeps your local settings and shows the changes to the code for you to decide whether to upgrade.
This script is a Canvancement -- a Canvas Enhancement. The links in the document point to an installable version of the code, but there is an Access Report Data project page as well that contains the source code as well as a version that you can use to anonymize the names like I did for the videos. Other projects, like the Roster Sorter that was mentioned here can be found on the Canvancement website as well.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.