Access to detailed Canvas logs in GUI

(4)

Provide access to detailed logging activity in Canvas courses in the Admin tools area and the ability to search by key data or words or download the data.

Right now we have only minimal information on user activity. Anything beyond changing course settings and we have to reach out to Instructure support.

I would like to have activity access logging such as the information below:

  • Any deleting or additions in a course
  • Any grade changes in a course
  • Any addition of new apps
  • Any users that are added or deleted
  • Any terms that are added or deleted
  • All SIS import batch files processed
18 Comments
cms_hickss
Community Contributor

For the "Any grade changes in a course" have you tried using the "Grade Change Activity" option available on the "Logging" tab on the "Admin Tools" page?

This will give you a grade history. I am going to say it's not the greatest as it doesn't allow you to download or sort the displayed information, and you also have to play the "scroll and load" game; but it will tell you who and when a grade was changed.

dholik
Community Participant

Yes I've used it a couple times but found in most instances I still needed to go to Instructure to get the detail I needed.

nancy_lachance
Community Contributor

I would also add that we have multiple instances when we need to determine "who did that..."  For example, a prof says someone mysteriouls changed an assignment in his/her course and says it just happened.  We really need to be able to look and see what actually occurred.... without having to go to the Help folsk to get an answer.

awilliams
Instructure Alumni
Instructure Alumni

Hi  @dholik ‌. As cms_hicksspoint out, there are a number of reports available for admins and those are discussed in more detail at How do I view reports for an account? It sounds like you have found those reports and are still seeking more detailed information. The request for more specific reports or access to information is not uncommon, and to address this concern Canvas released Canvas Data. Someone will surely correct me if I am wrong, but I believe all of the information you are requesting should be within this data. Canvas Data is designed to do just what you asked, "Ultimately we want access to everything that we have to reach out to Instructive support to get so we're more independent when researching issues or Canvas activity." Because of the wide array of questions and ways of reviewing data that may be needed, Canvas Data is provided as data downloads and you will be able to decide how to process and present that data. This does require some setup and will not provide you answers out of the box.

Canvas is also continuing to work on additional analytics tools and you can read more about those at Priority: Analytics 2.

Because this functionality exists in various parts in account reports and Canvas Data, I am going to mark this idea for archival. If you think there is still a part or parts of this idea that do not already exist, I encourage you to modify the description of the idea or create a slightly different one. If you change this one just leave a comment and mention me (How do I @mention people, places, content?) so I can review it and change the status.

cms_hickss
Community Contributor

 @dholik ‌, what piece of detail was missing for you? I ask because maybe it's just a matter of asking to have that added as a column in the "Grade Change Activity" log.

dholik
Community Participant

We have WAY too much data to comb through to try to find one thing like "Who changed that student's grade?". We do have RedShift now but querying that would be way to intensive for these types of requests.

Currently in Canvas under Admin Tools is an option for "Course Activity" what I'm recommending is that you expand on this feature to include more than changes to the course settings page and provide one or more areas to do a key word search of the data.

dholik
Community Participant

That would definitely be a part of it but the other I see a lot is who added or deleted this piece of content in a course whether it be a text or discussion thread.

dholik
Community Participant

What does new analytical tools have to do with my request?

I would recommend that we meet before you mark my idea for archival because so far the 'recommendations' you've provided are not acceptable or completely off base.

awilliams
Instructure Alumni
Instructure Alumni

Hey again  @dholik , creating a feature idea that complies with our guidelines in How do I create a new feature idea?‌ can be tricky, and that is why I am here to help. I will try to address your questions and concerns from both posts one at a time.

What does new analytical tools have to do with my request?

The Priority: Analytics 2‌ project includes a scope of work that would apply to Administrators. From my reading of this idea, it sounds as if you are an administrator looking for additional analytics features so I felt this would apply.

... so far the 'recommendations' you've provided are not acceptable or completely off base.

I am sorry to hear that my recommendations were received as off base. Through continued dialog I hope we are able to close this gap.

We have WAY too much data to comb through to try to find one thing like "Who changed that student's grade?"

In this case it sounds as if there is a very specific piece of additional information or functionality desired and that would be something that could move forward for voting. However, in the original idea description, the request is more broad. It includes a request to:

... access to everything that we have to reach out to Instructive support to get so we're more independent when researching issues or Canvas activity

and

the ability to search by key data or words.

To reference a particular section of the guide, How do I create a new feature idea?

Have one idea per submission:

Submissions that contain more than one idea can be unclear and confusing when it comes to voting and discussion.

If the idea can be modified to include a single idea that does not already exist in some way, I believe it could be moved forward for voting.

dholik
Community Participant

Where in my request did I mention analytics or even hint at statistical data that could provide trends?

I am sorry to hear that my recommendations were received as off base. Through continued dialog I hope we are able to close this gap.

How can we close the gap?? You provided one response to your assumption of what I asked then slammed the door in my face by archiving my Idea. So where in that is the dialog you hoped we'd have? You closed all dialog by shutting down my request without ever talking to me.

Which part of access to detailed logging activity was broad? Let me break it down a little more, I'm looking for a logging interface or dashboard in Canvas that administrators can use to look up details about activity that occurs in Canvas that we don't currently have access too which is the vast majority of what occurs in the system.

Also in some of the areas where there is logging activity that we can see it's a piled up mess. For example in the SIS Import area, we send hundreds of files through that area over the course of a couple weeks. How am I supposed to find the detail of something that occurred last week? There is no search, no way to target what I'm looking for?

awilliams
Instructure Alumni
Instructure Alumni

I am truly sorry that you feel the door was slammed in your face when I marked the idea for archival. I want to assure you that is not the case at all. It is part of our process to try to understand an idea and handle it accordingly. In some cases, this means pulling it out of voting. This does not mean it is permanently pulled out of voting. There is a good bit more about our process in How does the feature idea process work?

Now let's get back to the idea, because ultimately I want to make sure you feel successful getting your feedback across in the most productive manner. 

Let me break it down a little more, I'm looking for a logging interface or dashboard in Canvas that administrators can use to look up details about activity that occurs in Canvas that we don't currently have access too which is the vast majority of what occurs in the system.

As I understand it, with Canvas Data and Live Events in the Canvas Data Portal you can access such a "logging interface or dashboard." I am thrilled to hear you already have Amazon Redshift as I understand it that is a big step along the way with Live Events. I am not familiar enough with Amazon Web Services queues to know what the full functionality is when it comes to searching the large amounts of data Canvas generates for a specific grade change activity but we can find a lot more information about that if you are interested. I would tend to agree with you that the remainder of the challenge is around finding the right interface for both viewing the data in log form and searching it for answering questions.

I am going to change this idea to moderating so we can continue the discussion.

vanzandt
Community Champion

Hi awilliams, Please leave this idea, and others like it, open for vote. Canvas Data cannot be the answer for operational tasks; not in it's current form. We've been waiting for three months to get your team to provide us with a historical full data load, so that we can rebuild data that was lost after some odd event in December. The Data product and systems are just not mature enough to rely on yet. There is a serious lack of access to information in the Admin console and simple additions to the Admin Tools or Reports would go a long way to help us support your product. In addition to some of the items that  @dholik  noted, I would add audit logs of Canvas Admin changes (to better improve our security auditing), and better access to page view history, similar to View Notifications where you can specify date ranges (not the "last 300 item" csv download that is generally useless by the time you have to do research).

awilliams
Instructure Alumni
Instructure Alumni

Thank you vanzandtfor providing additional perspective from the Canvas Data side of things and additional use cases.

 @dholik , I believe I can see a version of this idea that would be acceptable to open up for voting. I am going to modify the idea slightly to remove the "ultimately we want access to everything" and to make the list of logging data a suggested list, rather than required list. This will prevent the need for a separate idea for every type of information that can be logged and it will also allow the flexibility for parts of the idea to be accomplished separate from others. Please let me know if you have any questions or concerns.

As with so many processes, the ideation process can be finicky at times, but I want to thank you and everyone else who aided in discussing this idea. As a former Canvas admin who never knew when the next difficult question was going to come from a faculty or administration member, I can entirely empathize with the need.

lamming
Community Novice

Hi,

 

We would like to have the ability to see who added (or deleted) a LTI-app.

 

I think that could be a part of this Idea.

 

Best Regards Mikael

jaleelja
Community Member

adding a report tool for "All SIS import batch files processed" would be great. We have more than one person in charge of enroll student in addition of Banner integration. I'm trying to find a way to extract/download the SIS import files uploaded by a certain user and/or know who enrolled a student and when.

Regards

brett_g_dalton
Community Novice

This is an issue for a lot of institutions. Recent example I had to examine was "who change this students enrollment.  Existing reports are problematic as they are generally fragmented, on a per-user basis which would have meant looking at the activity logs of every person who possibly had access to that course individually it still would not have included data from integrations.

Canvas Data is a partial answer and one I ended up using for this particular use case but it falls down in several areas

  • you need someone skilled at handling large scale data.  The request logs were 19GB for a system still not yet in production and less than a year old.  This is not the case for many small institutions.
  • Its not possible to check the issue immediately, only historically as Canvas Data is >24hrs old.  This is not useful or acceptable for security incidents and can be massively limiting for time critical events like problems with integrations such as Grade passback to the SIS.
Stef_retired
Instructure Alumni
Instructure Alumni
Comments from Instructure

The Reports page now includes the User Course Access Log report that provides key usage data for monitoring and reporting student online engagement. Please read through the Ready Release Notes (2020-04-25) for more information.

ProductPanda
Instructure
Instructure
Status changed to: Archived
Comments from Instructure

As part of the new Ideas & Themes process, all ideas in Idea Conversations were reviewed by the Product Team. Any Idea that was associated with an identified theme was moved to the new Idea & Themes space. Any Idea that was not part of the move is being marked as Archived. This will preserve the history of the conversations while also letting Community members know that Instructure will not explore the request at this time.