API Rate Limiting

Jump to solution
kenneth_robinso
Community Novice

I need an recommendation / solution.

 

When making Canvas API calls for one of our University Partner, if the University has 20K students (or more), we make 20K API calls to get each student’s LMS record.  Randomly, but often, we get “Unauthorized” and “Forbidden” response messages from the API calls.  And based on my research, it is because of API Rate Limiting that Canvas enforces (https://community.canvaslms.com/docs/DOC-8381-api-rate-limiting).

 

What is the best way to call the APIs if I want to get the LMS record for every student, for a given partner, in our database?

The end-game is to be able to successfully make a Canvas API call for every student in our database, for a given partner, in one thread.

 

I hope this is clear.

Would love to hear your input and recommendation.

0 Likes
1 Solution

I don't think your first take-away was what I was saying. Canvas has said that if you're making one call at a time, you won't hit the limit and it will basically stay at 700 (it refills faster than you can use it in a single threaded environment). That does not mean that you have to make calls one at a time, just that if you're only making one at a time then you don't have to worry about the rate limiting.

There are several API calls that will get multiple user records at one time. The one I use is the List enrollments endpoint. I get a list of courses and then fetch the enrollments for those courses. That works for me because I only need the students that are actually enrolled in courses for the purpose I'm gathering data.

I fetch the items in this order.

  1. terms so that I only process courses from current or upcoming terms.
  2. courses
  3. enrollments for each course. This gives me the section data, the user information, the grade in course, the total time in the course, and the last activity in the course.
  4. assignment_groups for each course (this gives all graded assignments with one call rather than multiple ones necessary with the assignments call).  I include[] assignments, but exclude_response_fields[] of rubrics and description and set the override_assignment_date to false (I don't need to mess with assignment overrides).
  5. submissions for each course using the multiple submissions endpoint. I use student_ids[]=all and a bunch of other parameters.
  6. student_summaries for each course using the analytics api

I guess only items 1-2 need to be first. The others could happen in any order. You'll notice that I get a lot information from the enrollments api.

The Canvas implementation of GraphQL is not as full-featured as some. Notably filtering isn't widely supported, so you cannot say "give me all students who have spent 0 time in the course."  I don't use graphQL much because what I need is available through the REST API. However, as Canvas moves forward, more development is taking place through GraphQL and not necessarily being backported to the REST API.

There is one API call that will give you a list of users if you don't want to garner it from enrollments or some other API call. That's the list users in account endpoint. It does not allow you to give it a list of user IDs (the search is limited to 1 item). It does not allow you tack on extra information that you would like to have -- that's why I don't use it for my data warehouse. I do use it in my nightly processing of SIS data to make sure that no records fell through the cracks.

View solution in original post

0 Likes