Community

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
RonniTyger
Community Participant

Script to pull "teacher" time in course and pull "student grades"

I would be willing to hire a consultant who can help us set up some scripts and teach us how to run them.  We are looking for a daily dashboard that the Assistant Dean could use to answer questions like, "Of all teachers with published courses in Canvas this term, who did not log into Canvas for 3 days in a row?"  Other questions we'd like answered at the end of a term by running a script using data include, "How many minutes did each teacher with a published course spend in the Canvas environment this semester?" and "What were the student course grades for each published course in Canvas that had a "teacher" that failed to log in 3 days in a row?"  This is an important project to us.  We have other questions.  We need a consultant.  We do have the Canvas Data Portal.  Let me know,  Summer is a great time to take this on. 

Labels (4)
17 Replies
jerry_nguyen
Community Contributor

@RonniTyger 

I can assist you with part of this project. However, there are limitations to which data we can pull from Canvas.

E.g 

Of all teachers with published courses in Canvas this term, who did not log into Canvas for 3 days in a row?

Data from Canvas Data Portal is always 24-48 hours behind (Instructure is releasing Canvas Data 2 which will reduce this delay to 4 hours). Thus we will need to rely on Canvas API to get real-time data. 

How many minutes did each teacher with a published course spend in the Canvas environment this semester

We can only get total activity time for a teacher if they open the course and do something /or stay in the course for more than 2 minutes. Other activities outside of the course won't be counted, E.g. if they simply open Canvas and go to the "Inbox".

 

Have a look at some of the reports I have created for our school here: https://community.canvaslms.com/t5/Vocational-Ed-Training/Assessment-reporting-amp-API-Canvas/m-p/49...

 

 

RonniTyger
Community Participant

Jerry, please email me at rtyger@tulane.edu.  I want to take the next steps. 

James
Community Champion

@RonniTyger 

Your questions indicate a real desire for quality instruction, but an unfamiliarity with Canvas (perhaps on the part of the person requesting the data). There have been some lengthy discussions here in the Community about the lack of accurate information for many of the things you're asking about.

You don't log into a course in Canvas, you log into the instance. A login may last an indefinite time depending on authentication settings (how long before you timeout a login?) and whether the teacher is using the app or the browser. There can be a lot of activity that goes on that isn't reflected in the last login date.

What you can measure, is what @jerry_nguyen mentioned: the activity spent in the course. Canvas documents that the user needs to spend more than 2 minutes in the course to pick up changes, but the number that it keeps is highly inaccurate. For example, it says I spent 606 hours in my statistics course last semester. That's almost 38 hours per week. I know I spent a lot of time in the course, but not that much. Even if you count the time that I put in before the semester started, it's still not that much. That means that the total_activity_time is pretty worthless for an individual, but can be used as evidence when other flags are present.

There is also a last_activity_at that shows the last time that an enrollment within a course was active. This sounds the most promising for checking whether someone has been active within the last three days. Except that this datetime is also unreliable. Some of the activity that happens automatically within the app when it is opened triggers an update to that value (at least it used to), so I would have students showing that they had been active, but their total activity time never increases.

What I found to be most reliable is to use a combination of the two. I gather the data nightly for each user and archive it in a database. Then, I look for the last date that the total_activity_time changed as the last activity. You will also need that tracking in place because Canvas Data isn't going to give you that information from a historical perspective. The last activity is worthless if a teacher came in the night before you checked but hadn't been in for 3 months prior to that. The total time in the course is worthless without looking at it historically because they may have spent a lot of time preparing the course to do little after that.

There are other things that can be done.

You can use the Access Report to see when they were doing things. Unfortunately, that's not easily available without being logged in to Canvas. I wrote a script that downloaded it for all students within a course and wrote a demonstration script that would do it for faculty, but it was really inefficient and tended to time out.

Much of the information that shows up in the Access Report (and some that doesn't) is available through Live Events. You can set up Live Events through Canvas Data Services that will send an event message each time a selected task occurs. I use it to see when users look at assets, submit quizzes, etc., but you could look at when grades are changed, conversations happen, discussions are used, submission comments are made, etc. This occurs in near-real time, rather than the delay of Canvas Data.

Live Events can miss stuff (it attempts to send the message exactly once if the servers aren't too busy), but that's typically not an issue, especially with something like what you're wanting to do. It also has stuff in that Canvas Data doesn't, but it provides events that you have to then operate on and accumulate yourself rather than getting summary data (through the API or Canvas Data).

You may also want to consider a pre-existing solution. The one that our college uses is made by AspirEDU ( @Chris_Munzo ). They are one of Canvas' partners and they have a product called Instructor Insight that does much of what you're asking for (they also have software that helps identify struggling students called Dropout Detective).

Jeff_F
Community Champion

We opted to not go with a vendor solution for reporting as we wanted flexibility and a solution built for our needs.  It is easy to show how Canvas Data provides a positive ROI.  Goodness, the reports that use CD are used by so many areas.
 
 
 
RonniTyger
Community Participant

Jeff, thank you.  These are the kind of reports we are looking for.  Our IT department has been working on a data mart and reporting for a few years.  We may have to go with an LTI solution like the one Aspire has called Instructor Insights or we may have to hire a consultant like you to help design reporting functionality with Data Portal and Banner (our SIS).  

Jeff, do you know how to get course or sub account level aggregate data about student viewing of Canvas Studio videos?  Each video has Insights.  We cannot collect viewing data one video at a time and all of the Studio Analytics seem to provide is data about upload numbers, storage, video length, and a few other things.  If you do , let me know and we can talk about next steps.

RonniTyger
Community Participant

Jerry, you are talented. Thank you for sharing. Academic Affairs is looking at an LTI solution that someone in this thread made me aware of. I will keep you posted if we end up going a different route and hiring a consultant.
RonniTyger
Community Participant

James, your recommendation to talk with Chris Munzo and check out Instructor Insight may be paying off for our school. Thank you for making me aware of an LTI solution.
James
Community Champion

@RonniTyger 

Canvas Studio uses a different API than Canvas itself. They call it the Arconaut's Manual, whose name made more sense before they renamed Canvas Arc to be Canvas Studio. Recently (this year), they added the endpoints for getting the viewing insights. Previously, it was a call to get pandata events. It's slow, you can only sustain about about one API call per second without running hitting the rate limits, but the information is there. It is not part of Canvas Data (that's a subset of the Canvas data that is available). There's no quick or easy way to get the data, but Canvas has said they are working to improve reporting. What they have now is a huge improvement over what they had in 2021.

About once a semester, I will gather statistics on viewership for my statistics course and share the data (along with some other metrics obtained from Canvas Data) with the students and we analyze it. I download the viewership data for the entire course, one video and one student at a time.

I do this by starting with a collection for the course. Then I get all of the perspectives for that collection, which is basically one per video. For each perspective, I fetch the insights for the course as a whole and then take the list of students who have watched the video and gather information about their usage. Then I put all of this into a spreadsheet so that I can manipulate it.

It doesn't cover everything you asked about in the Canvas Studio Viewing Data - course and sub account level thread, just what is available from the insights and by clicking on each student.

RonniTyger
Community Participant

James, I'd love to see the column headers on the csv file you use when exporting the various Studio Insights about viewership.