Obtaining and using Access Report data for an entire course

[James]

May 27, 2020, update (version 15):

Another week, another update. This time I installed a new linter to check for crud and subtle issues with the code. It turned up a lot of little things and at least one place where I was using an array when I thought I was using an object. Here are some of the changes for version 15

• The file is now sorted by the sortable name, when present. Version 14 sorted it by the Canvas User ID.
• Promises now work in the code. Version 14 relied on Bottleneck's idle function to kick in, but it might have triggered before all of the fetches were done.
• The script can now handle students enrolled in multiple sections of the course. You can generate a delimited list of sections, duplicate the information for each section, or just return the first section found.
• The script now only tries to fetch usage information when there is a last_activity_at date. That is no guarantee that there is an access report, but I never found an access report without having a last activity date.
• The script can now hide empty columns with no data. Previously, this was only for SIS data unless you edited the code.
• The script can be run correctly without reloading the page. Previously it appended the data, so if you ran it twice without refreshing the page, the data would be there twice.

May 19, 2020, update (version 14):

I've done a major rewrite of the script.

• It now uses a throttling library to play nicer with Canvas. Large courses were getting inconsistent results because the requests were failing. Requests shouldn't fail, but if they do, I throttle it even more and then try to repeat the failed requests.
• I added the Last Activity and Total Activity Time from the People page to the report.
• I added the current course score and current course grade to the report (the grade is only available if you're using a grading scheme).
• I modified the quiz views to match what Canvas gives. In the data supplied, they count taking a quiz as a view, but they subtract that off in the online access report. My data now matches theirs.
• I check the permissions of the user to make sure they can generate a report and don't add the button if they cannot. Previously, students could add the access report, it just didn't give them any information. Now it doesn't even show the button.

This is a major rewrite and updating will show a lot of changes. There is a good chance that any local customization you have done to the script will be lost. Those using a non-instructure.com domain for Canvas may need to reset their // @include line on line 5 (see the custom URL section below).

Synopsis

Canvas provides an Access Report for each student, but you have to click on each student's name from the People page to get it. This document will show you how to install a button on the People page that will create a .CSV file of the Access Report data for all students enrolled in the course. It will then show some ways you can obtain useful information from that data using Microsoft Excel. Although much of this data could be obtained through Canvas Data, this script makes it available to the instructor in real-time.

Quick Install

For those power users who are impatient, here are the quick install steps.

1. Install a browser userscript manager Tampermonkey for Chrome/Firefox/Safari
2. Install the Access Report Data user script.
3. Navigate to the People page and click on the "Access Report Data" button

If you run into problems, be sure to go back and read the instructions.

Note that the Tampermonkey extension for Safari requires payment. This is for the author of the extension that allows the script to run. I am not asking anything for the script itself.

Introduction

Canvas will give you an Access Report for each student that tells you how many times a student has viewed or participated in a particular content item. To obtain the list, you go to the Course Roster by clicking the People navigation link, then click on a student's name. Once you do that, you will get an item on the right-side navigation bar that says "Access Report for student's name". When you click on that, you get something like this:

It gives you the type of content (as an icon), the name of the content, the number of times a student viewed the content, the number of times a student participated in the content, and the last time the student viewed (or participated) in the content. It is sorted by the time the content was last accessed so you can see what they have been working on most recently.

Unfortunately, this information is available for just one student at a time, so answering questions like "Who viewed the PowerPoint presentation I told them to read?" becomes difficult -- you need to go into each student individually to see that information.

I've written a User Script that solves that problem. It fetches a list of all of the students in the class and then obtains the Access Report for each one, compiling all of that data into a single Comma Separated Values (.CSV) file that can be opened with a spreadsheet like Microsoft Excel.

This document shows you how to install that script and then analyze the data that you get from it.

Installing the User Script

A user script is a JavaScript that is ran by the browser on the user's machine. Rather than a Canvas-supplied script, it's one that the user installs and runs on their own. The installation is per user and per machine, so you will need to install it on each machine that you want to use the script with.

Custom URLs

The script automatically runs on any page that matches https://*.instructure.com/courses/*/users. This is the main People page if your site is hosted by Instructure without a custom instance. If you have a custom URL, like canvas.university.edu, then you will need to modify the script to get it to work.

The specific steps to do this vary depending on your browser add-on. In Greasemonkey, click on the Greasemonkey pull-down, choose Manage User Scripts, find the Access Report Data, right click and choose Edit. In Tampermonkey, click on the Tampermonkey Icon, choose Dashboard, and then click on Access Report Data.

In either case, you need to change // @include statement on line 5 to match your instance. In the case of canvas.university.edu, you should change it to https://canvas.university.edu/courses/*/users. The * is a wildcard that will match any course.

Customization

There are two configuration variables that can be set within the source code to alter the functionality of the script.

• showViewStudent will include the participations where a student clicked on a student's name from the People (roster) page and viewed their profile when it is set to true. The default is false, which will remove this information from report before the CSV file is downloaded. This information was included prior to February 3, 2017, but faculty were getting confused when they did a pivot table and the names of their students showed up as titles along with assignments and content pages. Now users will have to explicitly set it to true to get that information.
• quizParticipation is a flag to make the data in this report match the data displayed in the Canvas Access Report. Canvas counts quiz participations as views, but then subtracts them off when displayed. When this is true, it makes this report match Canvas. If false, then it displays what was in the JSON data, which was the previous behavior.
• enrollmentStates is an array that says what kind of enrollments should be included. The default is 'active' and 'completed', but you may also use 'invited', 'rejected', and 'inactive' (inactive probably won't help much). See the List users in a course endpoint for additional information.
• analytics is an array that provides an easy way to include the student analytics (last activity and total activity) or grade (current grade and current score).
• disableMissing is a Boolean value that will remove empty columns from the output when true.
• headingSpaces will allow you to remove the space in the headings and replace it with something else (empty string or underscore are common). This can cause "User ID" to become "UserID" or "User_ID". Some users reported problems with programs, like R, when there were spaces in the variable names (headings). The default is ' ' (a space), which is to leave the spaces.
• multipleSections determines how students enrolled in multiple sections are handled. Setting the value to 1 will duplicate the information for each section, so a student enrolled in three sections will have three times the data. This is important if you are using the section in your pivot table, but will throw off the counts if you are not. Setting the values to 2 will take the first section that it finds and use it and ignore all of the other sections so that each item is included only once with no duplication. Setting it to a non-numeric string will use that string as a delimiter but not duplicate any items. The default is ', ' (comma followed by space) so that multiple sections will show up as a comma delimited list.
• maxConcurrent is the maximum number of concurrent calls to make to Canvas. This is part of the throttling system. The higher this is, the faster the calls can be made, but the more likely you will abuse Canvas in a way that causes them to block your requests.
• minTime is a delay in milliseconds injected between each call. If you make a lot of calls that the same time, Canvas imposes a penalty that can quickly exhaust the available limit before the maxConcurrent is reached. It is likely that this value will have more impact that maxConcurrent will.
• debug is a Boolean flag to output debugging information. Included in the debugging output is the minimum x-rate-limit-remaining value and the maximum x-request-cost value. Those can be used to help tweak the maxConcurrent and minTime settings.
• csvFields contains the headings, source field, any special formatting required, and allows you to disable columns that you do not want. The three sources are u (user data), s (section data), and a (access report data). This is under the advanced configuration section, which typically means don't mess with it if you don't know what you're doing, but any user should be able to change the name property.

Export the Data

In the June 4, 2016, production release, Canvas consolidated the buttons that previously appeared on the People page under the administrative cog.

The script was updated on June 6, 2016, to reflect that change and move the Access Report Data into that same cog. Note that the demonstration videos have not been updated to reflect the change.

When you expand the menu by clicking on the cog, then you can choose the Access Report Data item.

Click on this and wait.

There is now a progress bar that appears once the list of students has been downloaded and it is fetching the access reports.

It has to make an API call for the list of students and then generate the Access Report for each student in the course. This only took 8 seconds for my class with 46 students and up to 112 content items, but we're only one month into the semester and it will probably slow down later. It took 12 seconds on a course with 81 active students and 126 content items. It took 15 seconds for a class with 35 students but 470 content items.

Limitations

Early on, really large classes were having timeout issues in Chrome. Switching to Firefox helped. Now I wonder if the problems were related to the requests failing and so I hope the changes I made with version 14 have improved this so that it is no longer an issue. I've made some improvements to the code with version 14 and let Chrome run for 4.5 minutes in a class with 29,615 items without issues

If you have a really large class, you may need to tweak the settings. The call to get the enrollments is relatively expensive compared to the call to get the usage (access report data). In a class of 300 students, it was fine, but in a class of over 10,000 students, it was timing out. It is supposed to retry failed attempts, but there was some data loss that I haven't tracked down yet. It's best to avoid exhausting the x-rate-limit-remaining header. Increase the minTime or decrease the maxConcurrent values. Normally, the minTime is the more important one, but if your requests are taking a long time to process (like the enrollments were), then you may decrease the number of concurrent requests. Reducing maxConcurrent from 40 to 20 avoided the lost data issue in the 10k+ course and only added 4 seconds to the download time for the class with 300 students.

The course with 81 students actually had 233 students in it, but it was a resource course for faculty and only 81 had bothered to go into the course to do anything. That brings up an important note about the Access Report.

The Access Report only provides information on students who are doing something in the course.

Access Report data does not return information about who has not done something, but Excel has an option that will allow you to see this.

Another thing to note is that additional information is available through the Page View data, like the exact times when students did something and which browser they were using at the time. However, faculty don't generally have access to this information and you have to load the information for each student and sift through it to see which applies to the particular course. This takes way more than 8 seconds for little gain.

The Page View information is available through Canvas Data. However the data there is not current, running about a day or two behind real-time. It also has limited availability and requires additional resources to analyze. Page view data still occurs when a student does something. If they're not doing anything, there is nothing to record.

Finally, links to URLs that are contained within assignments or pages are not included here. The External URLs are just those that are linked to items in modules, not a link in a page full of instructions.

Raw Data

Okay, you've installed the script, clicked the button, and opened the file in Excel. Now what?

Well, you get a bunch of raw data that will need manipulated before you can tell anything useful. It looks pretty intimidating, you have a report that looks like this (and this is just the first 10 rows).

Note that the SIS Login and SIS User ID may not be there, depending on the permissions you have within Canvas.

Where did all this come from?

It turns out that Canvas actually provides more information than it shows on the page you get when you ask for the Access Report. Canvas displays the HTML version with just the highlights and in a form that is easy for humans to read. It also provides the data in JavaScript Object Notation (JSON) format that is much easier for a computer to deal with. Canvas internally calls these items assets, and that gets reflected in the naming of the item.

Here's what that first entry looks like in JSON.

{
    "id":123456789,
    "asset_code":"discussion_topic_11319205",
    "asset_group_code":"topics",
    "user_id":1278402,
    "context_id":1785810,
    "context_type":"Course",
    "last_access":"2016-02-12T21:31:11Z",
    "created_at":"2016-02-10T20:12:45Z",
    "updated_at":"2016-02-12T21:31:11Z",
    "asset_category":"topics",
    "view_score":2,
    "participate_score":null,
    "action_level":"view",
    "summarized_at":null,
    "display_name":"Discussion 4: Hypotheses and Errors",
    "membership_type":"StudentEnrollment",
    "readable_name":"Discussion 4: Hypotheses and Errors",
    "asset_class_name":"discussion_topic"
}‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

What's being displayed in the HTML version of the report is contained are the asset_class_name, readable_name, view_score, participate_score, and last_access. The User's name is displayed at the top of the report, but the user_id (1278402) can be used to identify the student (in this case Cubic Dream) and the user script merges the user information and the access data together into a usable format for the user. By the way, all of the user data have been anonymized, but the assignments and access data are from my actual class. Cubic Dream isn't my student's real name and 1278402 isn't Cubic's real Canvas User ID.

Data Dictionary

In statistics, I emphasize the importance of having a good data dictionary. You need to know what the values represent.

• User ID is the Canvas User ID. Everyone has one. Most people won't find it extremely interesting, but in the case of multiple students with the same name, it can help tell them apart.
• Display Name is name of the student that is displayed in Canvas. It's normally in First Last format, like Cubic Dream or Skinny Record.
• Sortable Name may not be available to you. There is a feature that your Customer Success Manager can enable that will allow you to list names in Last, First format like Dream, Cubic or Record, Skinny. If it's enabled, this provides you with a quick way to sort alphabetically by last name.
• Category corresponds to the asset_category_type of the Access Report data. It includes things like announcements, assignments, collaborations, conferences, external_urls, files, grades, home, modules, quizzes, roster, topics, and wiki.
• Class corresponds to the asset_class_name of the Access Report data. It includes things like announcement, assignment, attachment, content_tag, discussion_topic, google_docs_collaboration, quizzes/quiz, student_enrollment, teacher_enrollment, and wiki_page. Generally, there is a one-to-one correspondence between category and class, but some things like the roster category are broken into student_enrollment and teacher_enrollments for the class name.
• Title is the name of the content. In the Access Report data, it's called the readable_name There was a display name as well, but it was sometimes blank and when it was there, it mostly matched the readable name.
• Views is the number of times the student viewed the content. For discussions, this would be the number of times they went in and viewed the discussion. This is the closest most people will get to the number of messages they read, but it's the number of times they went in, there is no guarantee they actually did anything past opening the page.
• Participations is the number of times the student participated. For discussions, this would be the number of posts they made.
• Last Access is the last time the student viewed or participated. It is converted to the local time according to the browser.
• First Access is the first time the student viewed or participated. It is converted to the local time according to the browser.
• Action is either view or participate. I'm not exactly sure on this, but I think it corresponds to the last access.
• Code is a unique identifier for each piece of content. It contains the type of content plus the Canvas ID. For example, assignment_8556874 identifies this as assignment with a Canvas ID of 8556874. That is not very useful for faculty, but if someone was trying to link all the data together for advanced reports, it may be.
• Group Code is another way of categorizing the data that most people will ignore. The group doesn't refer to Groups in the normal sense, but is some way to organize the data. For example, there is an assignment_group_### group code field that appears to represent which assignment group the item belongs to. That means you could, with additional information, break it down by whether the assignment was homework, exams, projects, etc., depending on your assignment groups. On the other hand, every single code beginning with wiki_page_ belonged to the same wiki_ group code (at least for my course).
• Context Type should be Course or Group and relates to the Context ID to determine exactly which course or group.
• Context ID is the Canvas Course or Group ID for the course. It is used in conjunction with the Context Type.
• Login ID is what the user uses to log into Canvas with. For us, it's their NetID, but it could be an email address.
• Section Name is the name of the section the student is enrolled in. This may be a delimited list if the student is in more than one section. See the multipleSections configuration variable for more information.
• Section ID is the Canvas ID for the section the student is enrolled in. This may be a delimited list if the student is in more than one section. See the multipleSections configuration variable for more information.
• SIS Course ID is the Course ID supplied by your Student Information System (SIS). This column may not be there if the person requesting the Access Report doesn't have access to the SIS information from Canvas. It's a permissions issue and the script runs as the person calling it.
• SIS Section ID is the Section ID supplied by your Student Information System (SIS). This column may not be there if the person requesting the Access Report doesn't have access to the SIS information from Canvas. It's a permissions issue and the script runs as the person calling it.
• SIS User ID is what your SIS knows the person by. For us, it's an integer that uniquely identifies the user. This column may not be there if the instructor doesn't have access to the SIS information inside Canvas.
• Last Activity is the last time the student accessed the course as explained in How do I use the People page in a course as an instructor?  It is converted to the local time according to the browser.
• Total Activity is the total time in decimal hours as explained in the Canvas Instructor Guide. Instead of giving hours:minutes:seconds, 3:12:15 becomes 3.21 hours.
• Current Score is the current numeric score for the student in the course.
• Current Grade is the current letter grade for the student in the course. This is only available if a grading scheme has been set.
• Final Score is the score the student would get if all ungraded assignments were given a 0. This is disabled by default.
• Final Grade is the letter grade the student would get if all ungraded assignments were given a 0. This is disabled by default.

Note that section information was added after the original script was released and are not included in the videos.

Drilling Down to Specifics

Some questions can be answered using just the raw data.

Format as a Table

If you're going to examine the raw data, you will want to turn it into a table first. To do this, go to Home > Format as Table or Insert > Table. This allows you to filter or sort on a column, which will greatly increase your productivity later.

Who participated in a discussion?

Let's say we wanted to know who participated (and how many times) in Discussion 3.

I purposely started this one off in a novice way, to show what people who may not be familiar with Excel can do. There are more efficient ways of doing this.

• You can sort by the title by clicking in the Title column and going to Data > Sort > A-Z. An easier way is to click on the down arrow on the Title heading at the top of the table and choose Sort A to Z.
• Doing that may overwhelm you with information so you can use Ctrl-F to find the one you want. But once I did that, it turned out that Discussions are in there twice, once as a discussion and once as an assignment. This is a problem when you have multiple contents with the same name, so sorting by title may not be the best solution.
• You can filter the Category (choose topics) or Class (choose discussion_topics). To do this, click on the appropriate heading at the top and un-check the ones you don't want.

Who viewed an external URL?

You'll want to sort by title to group the content together. You could use the Code to group, but it wouldn't be alphabetical.

I then used filters to select the external URLs. Those can be found under Category (external_urls) or Class (content_tag) and found the one I was looking for.

Show Me!

Here is a video walk-through of the three items mentioned here.

Summary Reports

Sometimes you want a broader picture than just what a student did on a single assignment. To accomplish these, you'll need to create a Pivot Table.

Create a Pivot Table

To create a pivot table, you need to go to Insert > Pivot Table and click OK.

Once you're there, you will probably want to drag the Display Name or Sortable Name down to the Rows so you can break things down by students. Every one of the items in this section starts off the same way.

Pivot tables allow you to insert slicers to quickly filter the data or time slicers to view data over a particular time period (you must have a date/time field to do this, but we have first access and last access to pick from)

Analyzing Discussions

This whole Access Report project grew out of a desire to know how many times students had gone into the discussions and at least viewed them.

What I'd like to know is how many times did a student view or participate in a discussion this semester.

1. Choose Insert > Pivot Table
2. Drag the student's name to the Rows
3. Drag the Participations to the Values. Excel wants to do "Count of Participations" instead of "Sum of Participations". You can change this in several places, but double clicking on the heading of the table is probably the quickest. Change it from Count to Sum and then change "Sum of Participations" to something else. You might want to use "Participations", but that's already used, so you have to pick something else.
4. That gives the participations for the entire course. You could add a filter on the right side, but a faster way is to choose Insert Slicer from the top. This gives a nicer interactive menu where you can immediately click and limit your data.
5. After looking at Participations, you can do the same thing with Views. Drag it to the Values box, where, thankfully, it comes through as a sum. I would change the title from "Sum of Views" to something else, like "View"
6. You can sort the data by clicking on the Row Labels pull down. The default A to Z and Z to A are for the data in that column, but you can choose More Sort Options and tell it to sort by another column
7. To break the report down by the discussion, you can add another slicer for the Title and then look at participations and views one discussion at a time.
8. If you decide you want to look at all of the discussions, then drag Title to the Columns selector on the right side. You'll need to turn off the filter on the Title if you do this.

Here is a video that shows all of that in action.

Filtering by Time Period

The Course Roster (People) page shows you when the last activity of a student was and the total amount of time spent in the course. It does this in alphabetical order. Unfortunately, last activity could be just logging into the course, it doesn't mean they did anything else once they got there.

As a side note, if you have installed my Sort a Roster Canvancement, then you can click at the top of any column to sort by that column. You can find sort by section, by the time they last accessed the course, or even by the total amount of time spent in the course.

You can filter the information in the Access Report Data spreadsheet by time.

1. Choose Insert > Pivot Table
2. Drag the student's name to the Rows, and both Views and Participations to the Values. As before, change the heading "Sum of Views" to just "View" and the "Count of Participations" to Sum instead of Count and then change the title to "Participation".
3. Click on Insert Timeline and choose Last Access.
4. Change the slider from Months to Days and then highlight the date range to restrict the report to.
5. You could add filters (through the Slicers) to limit what kind of activity you want to look at.

Remember that only students who have data for that time period will show up. There is no easy way to get a list of students who aren't doing a particular thing.

Here is a video that walks you through the report.

Quick Tables

A quick table is a table that shows you information about a particular area. What we're going to do here is break down the information by the Class. You could just as easily choose the Category and some might find it more useful.

1. Choose Insert > Pivot Table
2. Drag the student's name to the Rows, the Title to the Columns, and Class (or Category) to the Filters.
3. Draw Views and Participations to the Values. Rename Views to be V (yes, just a single letter). Change "Count of Participations" to Sum and then rename it to just be the letter P.
4. Rotate the titles in Row 4 so that they are vertical. Do not just click on Row 4, it doesn't work; you need to select the cells and then do the rotation. To rotate the text, click on the Orientation icon from the Home screen and choose Rotate Text Up. You may also want to Right align all the text from columns B on, although it may not really matter if you use a single letter for Views and Participations.
5. Be prepared to be wowed!
6. Click inside the pivot table, then at the top click on PivotTable Tools > Analyze. On the left, choose the pulldown menu next to Pivot Table Options (don't click on the word Options). Then click Show Report Filter Pages (this won't be available if you forgot to put something in the Filters box on the right). Then choose the filter(s) to use and click OK.
7. What you get is a page for each type of content. The name of the student is on the side and the name of the content is across the top. This allows you to quickly (hence the name Quick Table) look at discussions or quizzes or external URLs or anything else in the class that is available.

Here is a video that shows all of this in action.

Viewing Who Has Not Participated

As mentioned above, what we are looking at is the Access Report data and it doesn't include information about students who are not engaging in your course. Luckily, there is one checkbox in Excel that we can check to get that information. The student will have needed to do something, anything, so that their name is in the Access Report data, but then we can see what they have not done.

1. Choose Insert > Pivot Table
2. Drag the student's name to the Rows
3. Drag the Participations to the Values. Excel wants to do "Count of Participations" instead of "Sum of Participations". You can change this in several places, but double clicking on the heading of the table is probably the quickest. Change it from Count to Sum and then change "Sum of Participations" to something else. You might want to use "Participations", but that's already used, so you have to pick something else.
4. Choose Insert Slicer from the top and add slicers for Category (or class) and Title. Choose the content you want to view. What you currently have is a list of those who have participated.
5. To get the list of those who have not participated, click on the student's name from the Rows field and choose Field Settings. Then click on Layout & Print. Check the "Show items with no data" and click OK.
6. If you like, you can go to Data > Sort > A to Z to bring those who have not participated to the top.

Here's a video showing how it works.

Updating the Script

Tampermonkey should attempt to update this script automatically for you. In case you have turned off that functionality or would like to update sooner, you can click on the Tampermonkey icon and choose Check for userscritp updates.

The update process keeps your local settings and shows the changes to the code for you to decide whether to upgrade.

Canvancements

This script is a Canvancement -- a Canvas Enhancement. The links in the document point to an installable version of the code, but there is an Access Report Data project page as well that contains the source code as well as a version that you can use to anonymize the names like I did for the videos. Other projects, like the Roster Sorter that was mentioned here can be found on the Canvancement website as well.

[a1222679]

I looked into this a little more and found this today. 

As you can see with this student in the same course, the correction applies to the first two but none of the others.  All of these have the Group Code “quizzes”.  Some of them were corrected for the student above but not here.  In both, once the correction stops working it doesn’t restart.

COURSE ACCESS REPORT		Title	INDIVIDUAL ACCESS REPORT
Times Viewed	Times Participated		Views	Participations
37	17	lecture test 3 mc	20	17
7	2	Week 8 Practice Quiz	5	2
14	5	Lecture Test 2 Multiple Choice	14	5
4	1	Week 7 Practice Quiz	4	1
8	3	Week 6 Practice Quiz	8	3
5	2	Week 5 Practice Quiz	5	2
9	3	Week 4 Practice Quiz	9	3
3	1	Field Trip - Safety Information	3	1
3	1	RSVP for Optional Field Trip	3	1
4	1	Lecture Test 1 Multiple Choice	4	1
3	1	Week 3 online questions	3	1
9	2	Week 2 Online practice questions	9	2
6	1	Plate Boundary Practical Test for marks	6	1
8	2	Week 1 Online Practice Questions	8	2

[James]

 @a1222679  

I don't know Ruby well enough to debug how that corrected_view_score function fits in with the rest. The code is a little convoluted, seemingly making corrections that aren't necessary (subtracting 0), but I guess that allows for other corrections in the future, so I kind of understand why they did it that way.  

I wonder if possibly participate_score starts off as an integer but gets converted to a string at some point. I noticed that some of the participation_scores were showing up as decimals now instead of integers. What if that really wasn't a decimal, but a string representation of a decimal? Example: "2.0" instead of 2.0 ?

I don't know Ruby, but I found a script that checked for truthy vs falsey in Ruby and modified it a little to check to see what was happening with the ||= and the -=

def deduct value

  puts "#{value.inspect} for value"

  deductible = value || 0;
  puts "#{deductible.inspect} for deductible"

  value -= deductible
  puts "#{value.inspect} for diference"
  puts ""

end

[ 0, 1, 2.0, 3.0, "0", "3"].each do |value|
  deduct(value)
end
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

When I execute it (took a wild stab in the dark on how to do this, it was just ruby filename) I get this:

0 for value
0 for deductible
0 for diference

1 for value
1 for deductible
0 for diference

2.0 for value
2.0 for deductible
0.0 for diference

3.0 for value
3.0 for deductible
0.0 for diference

"0" for value
"0" for deductible
test.rb:8:in `deduct': undefined method `-' for "0":String (NoMethodError)
        from test.rb:15:in `block in <main>'
        from test.rb:14:in `each'
        from test.rb:14:in `<main>'
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

The place when I sent it a string that looked like a number. "0" (the string) is not 0 the number. In Ruby, strings are considered truthy, which means that "0" already has a value (the string) and so deductible is not assigned the value of 0 (the integer), it keeps its value of "0" (the string).

Then, when it tries to subtract the deductible, it's trying to subtract a string, which isn't defined for strings and it throws an error. But we never see the error, it just doesn't change anything. 

Again, I don't know enough about Ruby to tell if that's what's going on or not, but it might be a typing issue. Perhaps there is a casting or conversion going on that is making it not a number? Maybe there was a change between June 8 and June 10. It wasn't a scheduled production release, but sometimes they apply hotfixes.

I think I've tracked down when it happens, though. It looks like the ones with the decimal are the ones that are being subtracted.

Here's what my filtered access report looks like inside Canvas.

Here's the same thing with the results from the JSON data side by side.

	JSON		Report
Content	View	Participate	View	Participate
CH3 Q2	6	1	5	1.0
CH3 Q1	9	2	7	2.0
CH2 Q5	6	2	4	2.0
CH2 Q4	8	3	5	3.0
CH2 Q3	8	3	5	3.0
CH2 Q2	8	3	5	3.0
CH2 Q1	13	5	8	5.0
CH1 Q6	5	2	3	2.0
CH1 Q5	12	4	12	4
CH1 Q4	7	3	7	3
CH1 Q3	9	3	9	3
CH1 Q2	13	3	13	3
CH1 Q1	27	5	27	5
Grades Survey	4	1	4	1

That table was generated from the raw JSON to double check, to make sure it wasn't something with the spreadsheet.  

What I found was that all of the values in the JSON file were coming through as integers. There are no decimal points given. Here's the entry for CH3 Q2. Note the participate_score in the JSON is 1, not 1.0, but neither is it "1".

Since my script doesn't use the data from the page, but uses the JSON data instead, there is no way for me to match the inconsistency of Canvas. I could either subtract the participations for all of them and have it not match the ones in the report without decimals or I could not subtract anything and match the ones without the decimals.

At this point, though, I would consider it a bug in Canvas since it sometimes subtracts the participations and other times it doesn't.

Can you confirm that the ones with the decimal points in the individual access report are the ones that are getting subtracted?

I don't know if you manually pulled your results together or used a script, but here are some of the tricks I used to get my data.

To get the raw JSON, add .json to the end of the individual access report page. So if it's normally /courses/123/users/456/usage, change it to /courses/123/users/456/usage.json

What I did was to take the raw results from that and copy/paste it into the first line of this code. When I run it, I get the views, participates, and the name dumped in a tab-delimited form.

var d = copy_paste_json_output_here ;
for (var i = 0; i < d.length; i++) {
  var asset = d[i].asset_user_access;
  if (asset.asset_group_code === 'quizzes') {
    console.log([asset.view_score, asset.participate_score, asset.display_name ].join("\t"));
  }
}‍‍‍‍‍‍‍

To hide everything but the quizzes from the individual access report, open the developer tools in your browser (usually F12) and then paste this into the Console.

document.querySelectorAll('#usage_report table tbody tr:not([class*=quizzes])').forEach(function(e){e.style.display='none';});‍

[a1222679]

Hi James,

I found the same as you.  Where participation is a decimal in the individual access report the correction works but otherwise not.  So it does appear to be a bug within Canvas.

[fosterl]

An instructor ( @pugatcht  at my own institution just quoted me in this thread, wishing I had given the answer (which I actually do, further down in the thread). But in his case, I just set the course Ends date to a date in the future, and it worked for him because there are students. So maybe the way we (including  @kroeninm ‌ here at OSU) conclude courses (by term) has something to do with it. I dunno, just reporting what worked today for this instructor (in Chrome, BTW. Neither of us could get the script to install/run in Firefox ESR).

[James]

Greasemonkey has stopped working with Firefox, but Tampermonkey should work with Firefox.

[fosterl]

Thanks very much, tampermonkey works like a charm on Firefox ESR (as does your javascript). We're back in business  @pugatcht ‌

[jet4]

What if the "Access Report Data" icon doesn't show up?

I downloaded Greasemonkey and the Access addons to Firefox.

I downloaded Tampermonkey and the Access addons to Chrome.

I closed both browsers and re-loaded them. Both addons were successfully installed.

I went to People, and looked for the Access Report Data icon. It's not there in either one of my browsers. My course is active, and I am able to see the Access Report for individual students.

This is what it looks like in both Chrome and Firefox:

This is what I can see for an individual student:

[James]

 @jet4  , thanks for trying to use the script. Fortunately, I can answer your question in less time than it took you to write it.

First, dump Greasemonkey, it doesn't play nice since Firefox 54+. Use Tampermonkey for either Firefox or Chrome or Safari.

Second, read the quick install tips, the last line of which is: If you run into problems, be sure to go back and read the instructions.

The instructions you're missing are under Custom URLs. Since Rice uses canvas.rice.edu and not rice.instructure.com (or something similar), the script isn't picking it up. Change line 5 to match your institution's URL. To edit the script, go to the Dashboard in Tampermonkey and click on the script. Make the change and save it. Then reload your Canvas page.

[jet4]

Thank you! It's always something simple like that. 

It works now!

[josuecarames]

Hey James, thank you so much for this amazing script. I am trying to use the other script for instructors (canvancement/access-report-instructor.user.js at master · jamesjonesmath/canvancement · GitHub ) and I need a little help. So far I was able to install it in Tampermonkey and when I go to Admin and enter in my account, I can see the Tampermonkey recognizing that I have a script available to use there. The problem is that I do not know how to trigger the script and retrieve that csv document. 

Where do I need to be to trigger the script? Where would I find the link/button, etc. that will trigger that script? Thank you so much for all your work!!

[James]

 @josuecarames , 

That script is not supported. It was a proof of concept developed in February 2016 and the comments surrounding that are available in this thread near the bottom of page 1 and the top of page 2. In particular, there's one message where I explain all the reasons people shouldn't run it and then wrote:

I don't want to provide a direct link here because I want people to consciously decide to install it and understand the ramifications of just what that means.

Since then, Canvas has moved things around and so the button no gets put on the page, the FileSaver library has moved locations, Greasemonkey has stopped working with Firefox, and Chrome still has a timeout for scripts. Basically, it is out there for historical reasons if someone wants to build off of it, but I don't have the time or drive to fix it when it was marginally working at best.

If someone was really inclined, they could go back through the history of the main script, see how they differed, and then port those changes into the instructor one, but it really is terribly inefficient and most likely won't work unless you have a really small sub-account with not many classes in it.

[paulwuhj]

New to Access Report Data, after installing Tampermonkey and the script, can get the link and when click it is running, however, it never ends. And there is no download file to be found in the Download folder, which is supposed to be folder that the file resides. 

I did do the setting to correspond to our university URL. https://..../coureses/*/users

I also found some error messages in the scripts that were installed.

Do I need to install FileSaver.js also?

I actually also tried Firefox, using Greasemonkey, but I could not even get the Access Data Report tab to be shown?

What could have I missed?

Thanks,

Paul

[James]

 @paulwuhj  

And there is no download file to be found in the Download folder, which is supposed to be folder that the file resides.

This is a complete misunderstanding of what the script does. It opens a dialog box and then automatically starts a file download, all from the browser as soon as it reaches 100%. No file is saved in Canvas at all.

Greasemonkey is broken since Firefox 57 came out last year. It no longer provides access to JavaScript libraries that are loaded by the page, so it cannot access jQuery, which I use to make calls. Do not use Greasemonkey right now. Tampermonkey works with the major browsers, including Firefox.

FileSaver.js does need loaded, but it should be done automatically for you through the metadata at the top of the header. It is loaded just once, when you install the script. That said, FileSaver.js did reorganize their structure last year that broke the script. That's why I switched to using a CDN.  You can verify that FileSaver.js is loaded by going to the Tampermonkey Dashboard, editing the script, and then clicking on the Externals tab.

The warning icons you see are telling you that $ (jquery) or saveAs are not loaded, but they will be at runtime unless there was an issue installing FileSaver.js at the beginning.

Does it get to 100% and then stop or does it never get to 100%? It doesn't retry when a fetch fails, so it's possible that in a large class it might timeout.

You might try it with a small class. If it still doesn't work, you can go into the code around line 201 and add a console.log('SAVING'); right before the saveAs() line. Then open the developer tools using F12 and go to the Console. Reload the page in Canvas and execute the access report and see if it gets to that point.

[paulwuhj]

Thanks James for your ever helpful and prompt reply. I was out of my desk for the last few days. Just saw your feedback, after turning on the log, I realised I was not authorised to access the course anymore, although I had been the instructor for that course. And when I tried it on my current course, it does work perfectly, there is no error messages in the log anymore. 

Now, we are trying to understand the meaning of the fields. For instance, here are some fields and values in the output:

User ID	Title	Views	Participations	Last Access	First Access	Action
4552	Course Grades	4		14/5/18 17:30	27/3/18 1:23	view
4552	PCQ03	10	3	14/5/18 17:30	7/4/18 17:13	participate

What is the meaning for "Last Access" and "First Access"? Can we view this information from Canvas itself and if so, how?

Do they mean there 4 views in a duration of around 47 days, and 10 views and 3 participations in 37 days?

If this is the meaning, then it is not possible to identify the exact duration of each of the view and participation, correct?

And that information (duration of each action) would have to be derived/inferred from requests.txt table itself, I guess. Is this

correct?

Thanks again. 

[James]

 @paulwuhj  , there is a Data Dictionary section in the documentation at the top of this thread that explains what every variable means. The explanations may not make sense, but at least they're there.

First Access and Last Access are when the student first accessed the item and when they last accessed it. Access may be a view or a participation depending on the item. Some have both views and participations -- a student opened a quiz but didn't start it. Others are just a view -- a student read a content page.

The number of views is off by 1 for a quiz as Canvas subtracts the view used to take the quiz itself while I just return the information from Canvas without trying to manipulate it -- if I say 10, Canvas says 9.

You are correct that Canvas does not return the date and time of every access, just the first and the last. The web interface shows the last access, but not the first. It does not return duration information at all. Even the requests table is not going to give you good, reliable information on how long they spent looking at something directly, but you may be able to piece it together (they started looking at item A at 10:00:05 and at 10:00:08 they loaded item B, so you know they didn't spend very long on item A).

[paulwuhj]

James,

Thanks for your always helpful answers.

The current script reports on frequency, first and last usage/participation of an user. I have the following questions:

(1) In the synopsis, you mentioned the information is available from requests. More specifically,

     would it be appropriate to say that the two reports should tally as far as total number of views

    and participation is of concern. That means if you count all the URLs in the requests.txt pertains

    to a users, it should be the same as the sum of frequencies of view and participation. Except the 

    omission of certain counting as you indicated in your reply above. 

(2) Is there a similar script to report on the message content of each of the participation

     I presume Participation means the creation of some content, such as a forum message,

     while clicking the URL is considered a View). Is this correct?

I would also appreciate pointers to the documentation of the script in this thread or anywhere in the internet?

Regards,

Paul

[James]

I suspect that "participation" comes through the requests table as a PUT or POST command while the "view" comes through as a "GET". At least that's what I encountered in the limited testing I did. I certainly didn't test all the application and controller combinations tehre are.

I've not found reliability in the requests table that other people have. Over time, it seems that mobile is not included, then mobile is included, then every mobile API call is included, then it's not. Since questions in quizzes are auto-saved, there are lots of put or post commands that are made. If you were able to specifically identify which combinations matched up, then they might match.

Just now, I  went in to look at a recent quiz, I see calls to the quiz for take, submission events (through the API), backup, and show -- all with a POST -- yet that is counted as just a single participation. That one student shows 8 views and 1 participation, but there are 12 entries in the requests table. 6 of those are for backups and 3 three submission event requests. Part of the problem is that I don't have Canvas Data for July 3 yet and that's the last view he took, although he only took the quiz once as indicated by the quiz moderation page and the participation count. This student shows up with just one entry in the access report -- for the quiz. The other students have an entry for both the quiz and the assignment.

In other words, that information might be in the requests table, but I haven't explored it enough to say definitively that it is or isn't or when it is and isn't.

[ProfessorBeyrer]

I have been using this off and on and love it! I noticed that the filename for the  download is always the same, and I opened up the Tampermonkey editor to see if that could be changed. I found the saveAs command in the script and by demonstrating the principle of "fail fast, fail often" was able to append the course ID to the filename:

var saveFile = 'access-report-' + courseId;
saveAs(blob, saveFile + '.csv');

I inserted the saveFile variable definition within the makeReport function, which begins at line 188 in the script.

If I'm indiscriminately downloading access reports for multiple classes at the end of the semester (in anticipation of all that time I'll have to do analysis), it will be helpful if the courseID is part of the filename.

[paulwuhj]

James, 

Thanks for your feedback and comments on the first question I had. Based on what you said, despite the ambiguity in correlating view and participation with requests, the meaning (and frequency count) of views and participations is firm and can be used to measure students'

"intensity of the engagement" (through a proxy of the frequencies). If that is the case, do you think there is a way to "timestamp" these 

"views and participations" by correlating them with "bursts/clusters" of requests? 

I think you missed my second question, which I think is equally critical in measuring the "Intensity of the engagement" by looking at the message content of the participation if it concerns forum/chat/page content created/posted.

"(2) Is there a similar script to report on the message content of each of the participation

     I presume Participation means the creation of some content, such as a forum message,

     while clicking the URL is considered a View). Is this correct?"

Thanks in advance for your comments/feedback.

[James]

 @paulwuhj ,

My first paragraph was actually an answer to the second question. I do not know of a script that exists so answered the part about what constituted a participation. Looking at the source code while responding to this message, it looks like create or update is the definition of participation and those correspond to POST and PUT respectively.

Thinking about it, a script to do this would be pretty involved to write. In the case of a discussion, you could retrieve the contents of a discussion and look for a post by that user with that timestamp. In the case of a submission, you could retrieve the submission, but you need to watch out for multiple submissions since Canvas only returns the latest by default. In the case of a quiz, you can look at the submission details, again paying attention to the attempt number. Note this isn't necessarily true in Canvas Data, but it is available through the API. If the participation is editing a wiki or content page, you can look through the revision history and hopefully find the revision made by the student. You can also retrieve submission comments and get the content that was sent. Those are the places that jump out as me as potential participations in the Canvas source code or by looking at the access report. There may be other things that count, like collaborations or chat that I just don't use in my classes.

You can play around with Canvas Data and the requests table and see what you can get out of it, but I'm not going to be able to help much with it. Our current server is so old that it takes too long to do anything with the requests table (It took 13 hours last week just to load the full requests table into the database -- that was after 7 hours to load the rest of Canvas Data). We may move into that area, but right now, it's just too slow for me to do any analysis of the requests table (I met with our CIO who is going to look into getting me a faster server). You may want to ask your questions in the Data and Analytics group where there are people who do this kind of analysis. I do what I can, but there are some things that are beyond my abilities and knowledge and I don't want to keep you from getting other's expertise by asking here where they may not be following.

From what little I have used the requests table, I have a very low trust for it. For several months (December to April, I think), we lost a huge chunk of the data. Then it magically reappeared. Or maybe it was never there and we just had duplicates of everything except for those months when it was really small. It's too big to know if I have duplicates or not (at least checking with our current server).

Some people have tried to clump things together based off the session_id, and that might be okay for a browser, but I don't know if mobile apps are a single session or it creates a new session every time you re-open the apps. I'm not sure that mobile information is included, or how much is included. I know at one time it wasn't included at all because that was a big feature request. It even made the list of priorities coming out of Khaki 2017: Priorities & Related Ideas‌. In March, Mitch Benson wrote a Khaki 2018 Update that said that mobile page views was being released in June 2018. Once added, I don't know if they will go back and add mobile page views to Canvas Data or if it will just be from that point forward. Often, no attempts are made to apply new functionality to old data and in some cases it is impossible because it just wasn't collected.

All of that said, if I was attacking this problem, I would try to narrow down which one of the entries in the requests table actually constitutes a participation rather than looking for bursts of activities. Bursts could be spread out a long period if they're taking a quiz and you would only count one participation, but they could make several discussion posts in that same amount of time.

Come to think of it, you may be approaching the question from the wrong direction. Rather than using the requests table to see when they're participating, it may be more useful to look at the things that count as participations (from the second paragraph in this post) and query those tables. They are certainly smaller. Sure there's the issue I mentioned with Quizzes, but Canvas Data doesn't fully contain submission data for quizzes (despite the existing of the quiz_submission_historical_* tables). Or at least I haven't been able to find it in there.

Another thing to look at might be Webhooks or Canvas Live Events. You can set up a system where Canvas notifies you when something happens. That won't give you historical data, but it can let you know in near real-time as it happens rather than waiting the 24-36 hours for it to be reflected in Canvas Data.

Something else to consider is the Canvas analytics API. There is an API call to retrieve user in a course participation level. It provides a count of page views for each hour and it lists all of the participations for the entire course. It doesn't provide a lot of detail, but it does contain the URL and you can look at that to see what it was. You could even use those URLs to get patterns that could find the participations in the requests table, which goes with my comment that finding the right URLs might be better than looking for bursts of activity.

[paulwuhj]

 @James ‌

Thanks for the detailed discussion (particularly, on requests).

It seems I need to clarify what I mean by "script to report message content". What I meant more specifically was "Javascript script," like that for Access Report Data.

Based on API Documentation at Discussion Topics - Canvas LMS REST API Documentation,  we can fire an API, 

$.getJSON( "/api/v1/courses/8077/discussion_topics", function( data ) {
console.log(data);
});

We can then obtain all the topics are available on a particular course and then from there to go into each entries, then each replies and components of the entry. And

one by one, "crawl up" the entire discussion thread for example. By the same token, we could write the script to crawl page, file, assignment, etc. other message content. 

Using API, instead of Canvas Data, would allow us to do real-time reporting and under a specific context to limit the time required to generate the reports - all the benefits taht you foresaw in Access Report Data. 

Based on the above, I presume there is no such script available. If that is the case, would you be interested in jointly developing one (I must confess, I am at the basic level of Javascript only).

PS. I would like to continue discussion on requests.txt later after the above is resolved. 

Thanks, as always. 

[James]

Using API, instead of Canvas Data, would allow us to do real-time reporting and under a specific context to limit the time required to generate the reports - all the benefits taht you foresaw in Access Report Data.

Doing this in real-time while a user sits and waits for the results is going to be very difficult if your context has a lot of students or a lot of content in it. Even with optimization of the queries and guessing what the next link to handle pagination and use concurrent parallel requests rather than sequential ones, this is a lot of data to fetch.

I have two scripts that analyze discussions. The first does nothing more than count the number of responses each student made in each discussion for all discussions in the entire course. The second analyzes just a single discussion, but tries to provide particulars like how many words, when the first post was, the last post, etc. I wrote them in Google Script, with is basically an older version of JavaScript, but it makes synchronous calls and you need to wait for one to finish before starting the next. People have reported that with larger classes or lots of discussions, the first one times out. Google Sheets has a 5 minute timeout. The second one isn't publicly released yet, so I made it just a single discussion because it was timing out when I tried to do more.

Using a browser allows you 5-6 concurrent fetches, so things go faster. But people have reported here in this thread that Google Chrome also has a timeout on scripts and so Obtaining the Access Report wasn't working in some classes for them. I'll admit that I didn't know much of what I was doing when I wrote them, so there isn't any real optimization done that could speed it up.

Part of why the obtaining the access report script works is that Canvas pulls all this information together on the back end for you, so you don't have to download a bunch of information that you don't want just to get what you do want. When you download an assignment that happens to be a discussion, you get the body of the instructor's question twice -- once for the discussion and once for the assignment. There have been problems recently with really slow responses and lists of assignments or quizzes have been timing out when the information isn't already in the Canvas cache. That's in the core Canvas product, although some of it comes through the API as well. The first time I load a list of courses using the new admin interface, it takes 3 seconds for the first 15 to get loaded. After that it's faster, but it's a noticeable delay.

In some places, a single API call doesn't return everything you need. You have to make one call to get the information you need, then you make another call to fetch the information. The Obtaining the Access Report script begins by fetching a list of the students before it can ever begin fetching the information for the students. In other places, the only API call available returns too much information and you have to wait for it to download just for the tiny part that you do want.

I have no interest in writing or co-developing such a script. I can provide some limited support through the Community to what the API calls do, but it's not a project I would take on. You might join the Canvas Developers‌ group and see if any of them have an interest. I would clearly define the problem that you're trying to solve first, I'm still not sure what you're after or why and that makes it really hard to come up with the best solution.

There are companies that write programs that analyze data from Canvas, but I think for something like this that they download the data in the background so that it's available when you make the request. That is, it's not real-time, but "as of 4:00 a.m. this morning" or something like that.

[814068337]

For a data nerd like me, this is pretty nifty!  I am curious if any mobile app access is included in the data?  We always have to worry that students are accessing via mobile and we can't see it.  I'd hate to have a student have issues for not being present when, in fact, they'be been logging in and participating via the mobile app.  I will be playing with it to see, but figured asking is the easiest way to find out 🙂

[James]

814068337,

At one time, the answer was no. As part of the Khaki 2017 group, we prioritized including mobile data and Mitch Benson provided an Khaki 2018 Update in March that said it would be available in June 2018. It's now July 2018, but I don't know whether it has been included yet or not.

The question isn't specific to this script, though. If Canvas counts it, then this script counts it. If you know of a student who was using the mobile app to do something you could go into the Access Report for just that student and see if it shows up. If it shows up there, it will show up in other places. If it shows up, that means nothing really unless it was an area that you knew wasn't working before. Often when they put in something, it is not retroactive, so I do not know if page views from May will magically start appearing.

I don't remember seeing an announcement about it in June, but I will admit that I was busy with other things and not paying a lot of attention so I could have easily missed it. There is a feature idea for it that is still open and not marked as completed so they may not have finished it yet (or they might have missed closing it): https://community.canvaslms.com/ideas/9006-include-canvas-mobile-app-in-student-access-reports 

[814068337]

Wow, I appreciate the quick response! I will do some testing and follow up. I haven’t seen an announcement either, and since I know my faculty assume access reports include all methods of access, it would be great to know those were being counted (and I’ve been tuned in waiting to hear).

Lisa

[anthomak]

I can't overstate how much this may help me. I have spent hundreds of hours producing  resources for my students and made them available on Canvas, but never really found a way to gauge their usage. The Access Report is very revealing of student behavior (which shows I wish they accessed the resources much more), and having these data available allows me to justify my work to the administration, just because we now have a way to measure the outcome, at least in terms of usage.

The bonus is that you taught me how to use Pivottable as well. It is my first time using it, and it will save me tons of work in the future.

So here is my question...

I upload two main types of resources: PDF files and videos. I upload them to the Files tab (which is not available to students), and I create Modules which include links to those files. The Access Report works perfectly for PDF files. Whenever a student clicks on a PDF link in Modules which opens the PDF file inside the browser, the click shows up in the Access Report as "file --> attachment --> filename". However, when it comes to videos, it seems that simply viewing the video within the browser does not register in the Access Report. Specifically, this is what I found:

• If a student clicks on a video link in Modules and views a page with the embedded video, then the click does NOT show up in the Access Report.
• If a student downloads the video by clicking the download link above the embedded video, then the click DOES show in the Access Report, as "file --> attachment --> filename".
• If the video is an external link (such as Youtube), then the click DOES show in the Access Report, as "external_url --> content_tag".

My predicament is that my students access my videos mainly through the first bullet point. Is there a way for the Access Report to track those clicks? I guess the difference is that PDF are downloaded when viewed, but videos are just streamed when view, so I understand that if that is not possible. That means, if I want to track video usage, I will need a workaround, such as making my own Youtube channel, although that wouldn't track click counts once the student is outside Canvas (for example, if they click within the Youtube channel). Do you have any suggestions?

Let me express again my thanks for making this userscript! This is incredibly helpful.

[James]

 @anthomak , 

Welcome to the Community. Unfortunately, this script only rebundles information provided by Canvas, so all of those issues you mention are with Canvas itself, not the script. I can only report what Canvas makes available.

You are correct that external videos are not tracked by the access report. Including .mp4 files within the course itself isn't really a good solution from an accessibility perspective since they don't contain captions and they also take a lot of space. I tried an experiment this spring where I had a module (that I told the students to ignore) full of external URLs to play the video on YouTube and then linked to the module item rather than linking directly to YouTube. It was definitely a hack and not friendly. I ultimately gave up on the importance of tracking students watching videos.

Then we switched to Instructure's Arc software for our videos in May 2018. It is a paid solution, not part of Canvas, but it integrates with Canvas. It provides some tracking of views (it breaks the video into equal-length segments and shows whether the student watched each segment). It also allows commenting directly in the video. It also has a captioning service that, in my initial testing, is better than YouTube's in accuracy. At some point, we're supposed to get the ability to ask questions as part of watching the video. I would love to call that an endorsement, but I can't. It's interface is klunky and reporting is weak. Also, it is an LTI and the views do not come back into the access report, other than as accessing the external tool.

There are probably other services that will provide similar metrics, but I don't know of any that tie into the access report in a usable way.

[Shar]

Howdy Anthony,

Regarding your students primarily accessing the video by clicking on the link in Modules and viewing the page with the embedded video, wouldn't you be able to count the Page View as the student having access the video? So if every video is on it's own standalone page, the student going to that page means they went to the video.

Now whether or not students are viewing the videos is a different story, it's just as likely that a student would download a video to "watch later" and not actually watch it. As  @James  suggests using Arc can inform on what portions of the video were "played".

Cheers - Shar

[Maeve_McCooey]

Amazing, thanks for sharing James Jones' Blog. I'm currently looking at system reporting at the root account and the sub account level. Course level reporting seems quite strong (especially with this awesome tool) but I struggle when rolling the data up. 

[rodrigo_abarral]

106/5000
Thank you very much, it will help me a lot, but I would like to know if it is possible to also access the user's score

[James]

I'm not sure what you mean by "user's score." If you mean the grade on a particular assignment or the overall grade in the course, then yes that is available. But it is not available with this report as it is found somewhere else and not delivered with the access report data.

[rodrigo_abarral]

That's exactly what I was talking about because I need a report to unify this information. but anyway it is of great value this report I just solved a problem with a student using this information thank you very much for sharing.

[j_braak]

I'm running into something strange. 
In our test enviourment the script works. But when i want to use it in our production enviourment the csv wont download. It keeps 'loading' i guess.

Is there somehing i can do?

[James]

I'm not sure why that would happen unless perhaps your institution has some custom code that is modifying things so the script doesn't work or your test instance isn't a copy of your real instance (this might happen if you're self-hosted)

Does your production course have a lot of students or activity in it so that it's timing out? I think the script was fairly naive when it comes to error checking and if one of the calls doesn't come back with success, it could get hung. Is it broken for all courses or just a few specific ones?

You might open up the developer tools (F12) and check the Console for any messages related to the running of the script (open the tools and then start the download). There probably won't be any, but you might get a timeout or on the Network tab.

I did confirm that the script is still working in our production instance, so it wasn't a change that Canvas made that I didn't know about. Because it's not happening to me, it's also harder to track down.

[j_braak]

Hi James,

I do think the institution blocks the script because the Console shows me that im Unauthorized.

Thanks for the help!

[acohen11]

Be sure that you are enrolled in the course. I too had a similar issue, and it was because I was not considered an instructor in the course.

[acohen11]

I am wondering is anyone who is using this data, have you figured out a way to easily filter out Inactive students?

[James]

 @acohen11 ,

The API call I'm using to fetch the list of students (line 71) doesn't include their status. The only way to filter them out easily would be to change the API call used to get the list of students.

I think I picked that particular API call for a reason. If you use the /api/v1/courses/:course_id/students call, it doesn't tell you what section they're in. If you use /api/v1/courses/:course_id/enrollments, it tells you a lot more, but you can filter by state. You would also need to modify the handling of the list of students in the getStudents() function (line 92).

[lnadolny]

I'd love to use this but I don't have a cog, just the three dots and no "access report data" button. Anyone else have this problem?

[Shar]

Hi Larysa Nadolny,

And you modified the script in line 5 to point to canvas.iastate.edu?

Custom URLs

The script automatically runs on any page that matches https://*.instructure.com/courses/*/users. This is the main People page if your site is hosted by Instructure without a custom instance. If you have a custom URL, like canvas.university.edu, then you will need to modify the script to get it to work.

The specific steps to do this vary depending on your browser add-on. In Greasemonkey, click on the Greasemonkey pull-down, choose Manage User Scripts, find the Access Report Data, right click and choose Edit. In Tampermonkey, click on the Tampermonkey Icon, choose Dashboard, and then click on Access Report Data.

In either case, you need to change // @include statement on line 5 to match your instance. In the case of canvas.university.edu, you should change it to https://canvas.university.edu/courses/*/users. The * is a wildcard that will match any course.

Cheers - Shar

[lnadolny]

Thank you! I missed that step, but now it works. This shortcut is amazing!

[bzorella]

 @James ‌, these Canvancements are AMAZING.

The Access Report data has proven invaluable to our Faculty, but I have noted that since the October Canvas updates release notes that this userscript is no longer working in Chrome. It does, however, still work in Firefox? 

Same course, same People tab, same user logged in, TamperMonkey on both, same script version installed.

Chrome no longer shows the button, though Firefox does and exports the CSV as well.

The Console error in Chrome reports:

userscript.html?id=2f5c009b-4231-43db-be59-629e9dbd7138:2 ERROR: Execution of script 'Access Report Data' failed! $ is not defined

I have attempted to add a a setTimeout(function) to see if that helps, to no avail.

Is there any way to get this working in Chrome again?

I greatly appreciate all you've done for the community with these scripts, thank you!

[James]

This is a change within Canvas that is going to potentially break everyone using jQuery user scripts (the undefined $ is jQuery). It happened with QuizWiz as well.

I just became aware of it this morning when a message showed up in my inbox. I read something the other day that said Canvas was going to make sure that jQuery was available before the custom JavaScript was ran, but their attempts to make that happen must have also broken it for user scripts.

The question becomes whether it was intentional or a bug introduced in a deploy and it gets fixed quickly by Canvas. Basically, I'm going to have to rewrite all of my scripts to either 1) wait for jQuery to be available or 2) not use jQuery and write it in pure JavaScript. The second one is the approach I've been using in my recent scripts, but it's not a trivial undertaking.

I will get things updated as time allows, but depending on the script, people may have to live with the basic Canvas for a couple of days.

[James]

 @bzorella ,

I've modified the source code on GitHub to fix the issue. It's quick hack job to get it back up and going rather than a complete rewrite.

The part that was failing was the button creation. Because of the way that Canvas is now loading scripts (to speed up the process), the Tampermonkey user scripts are getting loaded before the Canvas scripts. That means that Canvas' jQuery is not available when my script runs and it wasn't able to add the button.

By the time you get to actually clicking on the "Access Report Data" button, Canvas has loaded jQuery and so I can use it for the rest of the process.

Once the script updates, you should be back in business. You can speed up the process by clicking on the Tampermonkey icon and choosing "Check for userscript updates."

[bzorella]

 @James ‌ you are an absolute LEGEND.

Thank you for getting this fixed so quickly, AND teaching me something new!

[lindalee]

Hey folks,

Our team regularly relies on  @James ‌' ah-mazing user access report for a variety of purposes -- thank you so very much! I can't even begin to guestimate how much time this has saved us, and what having ready access to this data has allowed us to do/answer/support.

However, we noticed something strange this week. One of our faculty members reported getting inconsistent results when generating this report:

I’m digging into the Access Report from Canvas that Linda helped me download the other day. I’m very confused because 1) it does not cover all my students, and 2) generates a different report each time I download it. So, it feels very unreliable and I’m not at all sure what goes into it. Can you please help me understand?

For example, the one I downloaded on Saturday had 58 unique students (of 128 registered), across both sections, and ~6500 total observations. When I saw I had only 58 students I downloaded it again this afternoon, this time getting 46 unique students and ~5000 total observations. I *immediately* downloaded the report a third time and this time got 58 unique students and 11,168 observations. Very confusing.

And when I used this report in a different class last night, I encountered the same thing. I had to generate the report three or four times before it included the specific student I was looking for.

Is anyone else seeing this kind of inconsistency? And any ideas about what might be causing it?

Best,

Linda

[James]

 @lindalee  

If I remember, it is not very fault tolerant of network issues (it doesn't retry failed requests) and if Canvas was acting up at the time, it could easily fail. The different number of students each time sounds like network congestion or slowness on Canvas' part.

The script was not designed for courses this big, but I don't have anything that big that I can test it on. My classes have few students (48 at most) but lots of assignments. It's downloaded one student at a time, so it's not the number of assignments causing the issue as much as the number of students. How many of the 128 enrolled students have actually done stuff?  Does the teacher have access to all 128 students?

Canvas was redoing the way the enrollments worked in April (I think), but I don't think I was using the shortcut they were working around, so it should continue to work. 

Can you take a look at the console log in the browser (normally F12) and see if anything is getting logged as an error message? Load up the console and then click on the button to obtain the access report. Also look at the network tab and see if the requests are timing out or coming back with an HTTP status other than 200 OK.

Finally, if the teacher is just after 1 student's information, it would be easier to go into the access report for just that one student.

Canvas did just (April 25) release their own version of this report, but it's only available to Admins and only allows you to go back one month from the current time.

The last time I ran the report was about 11 am on Monday and it worked for me, but I only have 44 students with 387 assets. There were 8898 total observations.

[lindalee]

Hi James Jones,

Thanks for your quick response. We've have definitely used your tool with some larger classes before without problem. But it's good to know that it's not fault tolerant of network issues and Canvas performance issues. Given where we are in this COVID-19 semester/year, that seems a very likely culprit. I'll take a look at the console log in the browser as you suggest and see if there's anything noteworthy there.

The time limitations around the New Analytics reports are frustrating -- a month worth of data, or a weekly interaction report often isn't as useful or convenient. 

Thanks again for the suggestions!

Best,

Linda

[linda_osullivan]

Hi  @James ,

First off if I can get this script to run it will be a major help to me in my work - I want to be in that data heaven so many have described.

Before I get stuck in setting it up - just one quickish question. Our institute has started using the New Analytics LTI so while you can still access the student Activity Report it's not as straight forward as you describe above, i.e. you first go to People and then to get to the Access Report, rather than the New Analytics report, you have to choose the Student Interactions Report, then select the name of a student and then you can view their reports which includes the Activity Report.

My question therefore is - will your script still work even if our installation defaults to using the New Analytics LTI. In my head, I think it should because the data your report accesses is still there in the background but I'm just concerned that because the navigation to the Access Report is so cumbersome from a user interface perspective whether your script needs to be aware of that or not.

To be honest, I'd prefer to access the data as you describe above as the reports that come from the New Analytics LTI aren't what I need for the analysis that I'm doing.

Any insight or direction you could provide would be so welcome,

Linda

[linda_osullivan]

 @James ‌,

Please ignore my stupid question above - I just stopped procrastinating, dived right in and installed the script and guess what it sooooo works!!!

I am now hoping to head in to very many hopefully fruitful few days of analysing my data and gaining some useful insights into content hotspots in a an online self help programme for students.

Thank you - and apologies for the ridiculous question,

Linda