Comprehensive Reporting on Faculty

[gibbonsd]

Has anyone created or generated reports focus on faculty members? Or faculty members activity?

Senior Leadership had requested a report to be generated indicating when and where the faculty members are accessing their course shell on Canvas during a specific time.   

I am generating a report based on the request from the Requests Table.  We are having difficulty determining information relevant to the course activity.   The report needs to focus on faculty members specific course activity.  However, there appears to be data information that does not reference any particular event.   Example, we see "activity_stream," and we have not been able to determine where this information comes from, how this "activity_stream" is triggered, and why this "activity_stream" is triggered.  

Is there a list of "activity" that would give a false positive about the faculty member activity being active inside the course shell when they are not?

[robotcars]

I'm curious about what you mean by 'where faculty is accessing', do you mean within Canvas or geographically?

[gibbonsd]

Robert, the Senior Leadership wants both.

[robotcars]

Hi Dona,

First, tracking a user to a specific time and location has some issues, see How to produce login and unique login reports

I'm working on a solution for this, but I'm keeping it close to the chest for now. Stay tuned.

Now, I know we've talked in another thread about Canvas Data, so I'm going to jump right in.

I have a process that includes Canvas Data, Ruby, a free API, and Tableau to generate the map in this tweet...

https://twitter.com/NVLearningAcad/status/1006302472371990528

I use the pseudonym table for Current Login IP (because there's too much noise in Requests). The API I use for geolocation provides everything to map the record and timezones for other purposes. It's a short Ruby script, but the whole process is wrapped around Canvas Data, querying it and then saving the geo info to a new table. I can share it on Github if you can support it or have a developer/programmer who can use it as a starting point.

As far as Where in Canvas the user is, I generally try to target some kind of context.

It's one thing to look for page counts by user role, course etc... Apr 18, 2018 1:31 PM

And another to map out the workflow the user took while navigating through Canvas, something like Sankey or Chord Charts (HE) Analytics in Course Design: Leveraging Canvas Data - Canvas Network 

Here you see the instructor and two staff members navigating through the course, these are page counts. The staff are jumping into the course and helping, the one on the right which I erased to clean up PII, shows that staff member coming in and adding Announcements to the course, you can also see that little thread from the Staff on the Left jumping in to add an announcement.

This shows us what pages they visited and how often, but not how long.

I find it's best to select a set of records or a type of record and specifically display that information with meaningful context. Below is a screenshot of our LTI Dashboard. The top chart shows time spent in the course by my best estimation using the Requests table. The second chart uses submission_dim and JOINS to try and determine how much time they spend grading assignments. These 2 charts don't really say Where the instructor was, but it does provide context when a School Admin wants to see how different staff spend their time. What something like this shows us after reviewing it for a couple semesters is that Teachers who do a lot of grading in SpeedGrader might have more time in the course or spend time grading, but teachers that have assignments that are graded obviously spend less. However, it's not necessarily an across the board comparison. Ex. A teacher may spend hours reading essays marking on them and coming back into Canvas and rapidly apply all the grades, which would show very little time.

I'd love to know what you're thinking or what your leadership wants to see, I'm always curious and ready to dig new rabbit holes.

[ColinMurtaugh]

Hi Dona --

Yeah, the requests table has a lot of detail that's sometimes tricky to interpret.  One thing to be aware of is that these days most web pages aren't delivered in a single request -- there are often secondary XHR or Ajax requests that are made to fetch certain contents of the page.  I don't have actual request data in front of me right now, but  the "activity_stream" requests that you're seeing may be loading some sidebar content, or perhaps the homepage content for a course that's displaying the activity stream by default. 

You should be able to filter out the noise by just looking at the rows in the requests table that have "Course" in the 'web_application_context_type' field.  You'll also be able to determine which course the user was interacting with by looking at the web_application_context_id field. 

Another thing that you might look at if you want to understand the user activity in more detail is the 'web_application_controller' column.  The Canvas application code has many different "controllers" that correspond to different functionality, and you might be able to use the values in the web_application_controller column to help figure out what part of Canvas the requests are associated with (e.g. assignments, files, pages, etc.). I expect that there'll still be some requests that are difficult to categorize this way, but it might get you partway there. 

I hope this helps -- let me know if you have any questions about anything above!

--Colin

[kona]

 @gibbonsd , have you looked into Instructor Insights -https://aspiredu.com/solutions/instructor-insight/  

Not sure everything that your leadership wants, but this product covers a lot of ground in terms of what activity is being done in the class and a comparison to the overall average as well. It might be worth looking into.

Kona