[Analytics] User Activity and Analytics reports for Instructors: Exclude Masquerade activity

[millerjm]

@For federal and state financial aid reasons, we need very accurate reporting of user activity.  It appears that the activity performed by an admin while masquerading is showing up on the user access report for the course and for the user activity on the people page, which are the places that faculty look to determine the last date of attendance for an  course. We know that Canvas Data has been released, but accurate user activity history needs to be in the hands of faculty as part of the normal business practice of reporting grades and last dates of attendance.

This idea is specifically to exclude any activity performed while masquerading from reports that instructors see, as well as provide easy access of information to admins about who performed these actions. 

Reports for Faculty

We are concerned because the reports that faculty see in the courses currently show any activity done while an admin masquerades as a user as the user's activity.  You can see the examples of this below in the screenshots.

As you can see below, it specifically states that this information would record as the admin performed them and not as the user. This essentially negates the usefulness of the “masquerade as user” functionality since it would adversely affect the user activity record and therefore the information that we report to auditing authorities.

In the documentation at How do I masquerade as a user in an account?, it states that:

While you are masquerading, you can act like the user. You will see what the user sees, but the audit reports will show that you performed those tasks and not the actual user.

These course-level reports are the only way that faculty can get a last date of attendance and this isn't accurately displaying since it looks like the student did this. I was chatting with support and he said that "You will see the mark when they begin masquerading in the users account on the user page views with a /masquerade."

However, these audit logs are only available to Instructure support and not to the Instructors or even institution admins.

After reviewing this information with some help from our next level support we were able to determine the documentation saying that we(instructure) would be able to check our logs and see who did the masquerading and that it was not the students but the person masquerading. Unfortunately this is not associated with reports that are run by the institution. It is in the wording of the documentation that the confusion can be made. when it says You can take any action as though you are the user but the audit logs will show that you performed the tasks while masquerading. This you is referring to Instructure. Your desire for the logs to reflect correctly the masquerading is a great idea and I would love to have you input a feature request for this here: https://community.canvaslms.com/community/ideas/feature-ideas

User report of last activity on People page (prior to masquerading):

 

After masquerading:

Reports for Admins

Additionally, we would like to request that the “Page Views” area on the users screen for admins by default exclude any masquerade activity.  This activity can be downloaded in the CSV file and is irrelevant in almost any case that we would be researching, or at least make it VERY obvious who performed this action.

Please note that I spoke with two support agents yesterday who told me that the audit logs are available only to instructure support and not institution admins but it looks like this information is shown on the CSV file download on the Page Views screen for admins.

[asatkins]

Yes please!! This is also SO awkward if there is a situation of an admin masquerading as another admin... the browser (an institutional release/update) will likely be the same and so will the IP, the only two 'easy'(ish) ways of telling. We had a sensitive situation with this last month and it was very difficult to tell or prove.

[admin_brake]

YES, analytics must be accurate for them to be valuable.

[lfeng]

Hi everyone - great ideas on this thread - thanks so much to everyone for sharing their thoughts!  I wanted to let you know Canvas Data has a backlog ticket filed to capture the real_user_id for reqeusts (page views).  Once that is there, any reports using Canvas Data can make use of this.  In addition, as part of our Analytics 2.0 project, we plan to surface this when we redo the existing Canvas reports and dashboards.

[millerjm]

Linda, thank you for the update! 

[Chris_Munzo]

Where can we learn more about the "Analytics 2.0" project?

[cward]

Hi everyone,

Canvas Data now captures the real_user_id in the requests data (page views).  Any reports using Canvas Data can make use of this, which enables you to filter out masqueraded activity.  In addition, we plan to surface this when we redo the existing Canvas reports and dashboards (date: TBD).

Thank you!

Chris

[ewerhardt]

While I appreciate this as a workaround, it's awkward at best.  Regardless, because we don't want to mislead an instructor or student by stepping in their shoes and creating page views, we'll have to stay away from Masquerading as a support practice.  This is disappointing.  We are new to Canvas and were able to do this in our previous LMS.  Workarounds aside, we would hope this fix is done sooner than later. 

[millerjm]

Hi Deactivated user​

I realize that canvas data captures it, but this information needs to be in the hands of our faculty.  We are very careful not to masquerade as students unless we absolutely have to, but I know that Canvas Support does, which makes me very nervous about the accuracy of the information that our faculty have in order to report last date of attendance for federal financial aid reporting... 

However, having it in Canvas Data does help us for the times that we get audited for this information...BUT when the admin level canvas data reports don't match what faculty reported in the SIS, and was originally reported, it calls into question data originally reported and could open up students to having to pay back financial aid money from a prior year...so many cans of worms here and unfortunately I know too well from experience...I would be happy to discuss with the team working on the redo of reports and dashboards. 

Thanks,

Joni

[ewerhardt]

Another worm can example ... At our institute, many faculty use course activity reports during the open add/drop period as a barometer of student's course engagement.  This tells them if a student may be at risk for failure.  Some will require students to engage in certain areas of the course by a certain date or be dropped.  So masquerading could potentially affect this outcome ... and telling an instructor to weed through a csv file each time, or have support or an admin do it, is just not an effective use of time.

[millerjm]

Yes, we must withdraw students who have not participated based on certain criteria as well, so it's more than just money at stake for these students.  And since the instructors don't have access to the CSV files, they are essentially unaware of this issue and there is no way that our staff would be able to provide this information with over 600 online sections a term.  If there isn't an indicator saying that someone did masquerade, they have no idea that they should be looking for anything out of the ordinary.