Welcome one and all, this document is designed to assist everyone from those that have never heard the API acronym before, to seasoned programming veterans who may be looking for tips and tricks that are specific to the Canvas API. I have worked to structure this document to get more complex as it goes on, so you should be able to find your comfortable starting point easily. Most of all, if something doesn't make sense, please leave a comment, if you have some ideas for inclusion, or even if you would like to challenge something that I have proposed, I encourage you to leave a comment!
So, you may be asking 'What is this API acronym'? API stands for Application Programmer Interface, and in this section, I will cover off some basics about what APIs are. When you access a website (such as Canvas), you are accessing an application. The application has a web interface that is written for humans to interpret information and interact with that information. These interactions may be adding data such as discussion posts, pages and module items or they might be changing data, such as updating student or staff information or enrolments in courses. This list is not extensive, and there are many other functions available. However, these are just some examples I will reference for highlighting the differences between the web interface of Canvas and the APIs.
The key difference between the Canvas web interface and the APIs is that the Canvas web interface is designed for a human to interpret and interact with the application, it presents information in a visually appealing way, using things such as iconography and visually logical structures to users such as staff, students, and administrators. The APIs differ in that they are designed for consumption by a computer. You can think of the APIs as 'just another website', but a website that computers can access to interact with Canvas and the information contained therein. For a computer to most effectively communicate with another computer, the visual elements and structures of a web interface such as Canvas' web interface, only get in the way. Therefore, the APIs present functions and data in formats that other computers and programs can call and consume in a standardised way.
|Canvas Web Interface Example||API Interface Example|
The APIs can achieve most things that are possible within the Canvas web interface. It is important to note that the APIs are for Canvas itself. Therefore tools installed such as LTIs will not be accessible through the Canvas APIs and are out of the scope of this document. The concepts herein, however, could be applied to any LTI vendor's APIs if they publish an API specification similar to the Canvas LMS API documentation.
The functions listed in the Canvas LMS API documentation are extensive, and some of the more common applications of the APIs include:
Indeed they can! Canvas publishes the full API specification, and any user can interact with the APIs. The key to note here is that a user will only be able to access as much information in the APIs as they are permissible to within the Canvas Web Interface. Therefore, if a student tries to query information about their account, they will have the authorisation to get the details back. However, if they attempt to run a query for another users' account, they would get an access denied error. Equally, I will reiterate here that a user must have an authenticated account or access token for your institution to access the APIs, without which, the API calls will fail.
A great way to illustrate this is the Canvas Mobile App for iOS and Android. The mobile apps use the same APIs that are available to institutions and users, to interact with Canvas and present information to users on the run based on their user account. The mobile apps take the information that Canvas presents in the API Interface Example above, interpret that information, and then re-present it to the user with visual elements much like the Canvas Web interface.
I am glad you asked! :smileygrin: There are some incredibly important things you should know before you get in and start playing.
There are several alternatives you may consider before you look at the APIs. I won't cover these in great detail, as there is a wealth of information already available, hence, I suggest you check out these links for more information.
So, now you know that Canvas has multiple ways to get data in and out of the system, you may be wondering which one you should choose!
One of the first things you must decide is how you connect to the API (i.e. with which type of authentication method). The method you choose will largely depend on your needs.
Canvas outlines the API rate-limiting policy in the API Rate Limiting document. However, when I first read this, it took me some time to get my head around the best way to approach designing my solutions to ensure I wouldn't run into any issues. I recommend having a read of that first, then come back here for some of my additional tips.
There were a few of things that we came up with as possibilities to ensure we did not run into any issues:
The option we have used for our major integrations are three and four. Personally, if you are doing anything other than sequential processing, I think it is good practice to gracefully check for, and handle the rate-limit error (i.e. option four). In our instance, we decided to run five threads in parallel, with the rest of the operations in those threads running sequentially. That gave us a good hybrid of performance, along with ensuring we would not hit any API limits as the most we have running at once is five API calls on a single token.
The next item you should be aware of is the need for pagination when you are using the API. Pagination is not overly difficult but is often encountered as one of the first stumbling blocks when getting into the Canvas APIs.
The Canvas API Pagination mechanism is documented in the Pagination - Canvas LMS REST API Documentation. So, what is pagination exactly? When you have large amounts of data that you are pulling out with the API, it would be incredibly system intensive to pull all that data out in a single call. One example includes using the API to list all courses in an account. You could conceivably have hundreds, if not thousands (if not possibly tens of thousands) of courses returned, depending on the size of your institution. If you made a single call and that retrieved all results, it could result in significant delay in even getting the results, and this is where pagination comes in!
Canvas (by default), only returns the first ten items by default in an API call and will also add an item in the 'header' of the return message indicating there are more 'pages'. While you can increase the per_page_limit, be aware that (as per the documentation) the per_page_limit maximum value is undocumented, so be aware, your mileage may vary if you choose to tinker with this. What you need to be most aware of with pagination is that no matter what you are doing with the API, no matter how simple, one-off, or just playing, you will need to handle pagination.
Once you implement pagination, things work a little differently. Using our previous example, when you call the API to list all courses in an account, Canvas will return the first ten and also tell you there are more pages. Therefore, your script/integration must recognise that, and then call the Canvas API asking for the next ten, and the next ten, and so on. For those starting out with programming, this is a great way to get into playing around with looping in your chosen language.
When you don't handle pagination, you will find you may struggle as you seem to get small result sets, you may think there is no real rhyme or reason to the result sets you get back from the API, and you may think data is missing. Therefore, if you run into any of these, I highly recommend ensuring you have handled pagination in some way, shape, or form, and that it is working correctly.
You may want to check out this thread Handling Pagination which has some interesting discussion on, and code examples for, handling pagination. I stumbled across this recently and it is a great starting point that I wish I had found earlier.
All righty, if you only read one section of this document (surely you want to read them all right?!?), please make it this one! I can not stress the importance of this concept, especially for those starting out with the APIs as things can go very very wrong, very VERY fast.
One of the great things Instructure provides for hosted clients is several copies of their production environment including Beta and Test. There is a slight downside of these the environments' regular cloning to be aware of, if you haven't been made aware of the better alternative (kudos to Deactivated user for calling out the update to this one). Historically, one had to manage developer keys for the Test and Beta environments manually. As Brett has now made me aware, that there is now an option when you edit a dev key to mark it as "Test Clusters Only" so it will only work in Test and Beta and not Prod. To use it, create the key in your Production environment and it will then sync down to the Test and Beta environments on refresh and be available for testing but wont work in Production!
The best practice option ensures that when you think you are playing around with a Test environment if you only have a key that works in the test environment, there is no risk of it connecting to your production environment, it would simply give access denied. When you have the same keys sitting on the production and test environments, you run the risk of accidentally thinking you are working with Test (or Beta), when in fact you are modifying production data.
There are a few more concepts you should know about in getting started in the APIs, for these I will refer you to the existing documentation already available, though if there are any that people would like me to expand on, please let me know:
When you want to get started, the best thing I can suggest is start small, don't try to build Rome in a day, build your foundational column and work on that until you get your head around it. Equally, don't think you are alone, there are some great resources out there to help you get started:
Figuring out your first task to tackle might seem simple, you have an immediate need, and you want to dive into that. I would propose something a little different. If you are starting out completely fresh, consider doing something you already know about, to simplify the learning curve. For example, you may need to automatically enrol some users with the API. Before jumping into this, I would propose taking the time to put together a script in your favourite language that does something more simple. By starting with functions and features you know well, the learning curve of how the API works will be easier to grasp. Some possible starting places you may want to consider include:
You can check out the source code for canvasapi at the official Github page --> GitHub - ucfopen/canvasapi: Python API wrapper for Instructure's Canvas LMS
Thanks so much for the recommendation of this video @audra_agnelly !
Once you have the basics of using the APIs under your belt, the next steps are programming for robustness. A lot of us start out with simple use-cases, looking to achieve a specific purpose. In such cases, a simple script that assumes a best-case scenario is often appropriate.
Moving beyond the simple scripts, however, you get into the realm of developing for reliability and robustness. I have prepared this section with some insights into things you should keep in mind when you start developing solutions for more than just yourself, or very simple tasks.
This may sound simple, and this may even sound incredibly logical, but when you start out, I implore you to log your actions and what your integration/scripts are doing. It is great practice to build in both basic and debugging logging. You may want to look at a high level of detail such as 'something happened that I didn't expect - log it', or you may choose to go down to the level of 'I am about to try something', 'I am trying something', 'I tried something and this was the result'.
By adequately implementing logging, this will (by its very nature) force you to implement excellent error handling. More on that below! As an example for Python, check out https://fangpenlin.com/posts/2012/08/26/good-logging-practice-in-python/ (Python 2) or The Pythonic Guide to Logging (Python 3, recently updated) which covers off some good logging practices. The reason I use this as an example is it shows the usage of a proper logging module, which does a lot of the heavy lifting for you. Once you have your language chosen, jump over to Dr Google and search for something along the lines of '<language> logging best practices' or '<language> logging modules and error handling'.
Something I have seen come up often surrounds API timeouts. If you work in an enterprise environment, you may have worked primarily with self-hosted systems, rather than something in 'the cloud' (or as the formal term goes, Software as a Service, or SaaS for short). When you have an application or system hosted by your institution, rather than over the internet, there are a lot more known factors, especially surrounding the network. When you jump over to a SaaS application such as Canvas, however, you have hundreds, if not thousands of network devices between you, and the system, therefore, you should develop your integrations with additional robustness.
A common misconception is that 'the API should not timeout, it should be more robust and I shouldn't have to handle timeout errors'. Today I am here to bust that myth! Any number of issues can cause timeouts of which Canvas is only one small piece of the network puzzle in the mind-bogglingly colossal chain of devices you rely on to connect between the two. Sometimes in the ether of the internet, something takes a little longer and some of what you sent over the internet goes astray. Most of the time, the internet is robust enough to handle this, sometimes, despite the best-laid plans of mice and men, you get a timeout.
Therefore, in my opinion, correctly handling timeouts is not even optional, it is a necessity. One great implementation is that if you get a timeout, you should try again, if you get another timeout, implement a mechanism to sleep and then try again after X period (e.g. 60 seconds). You may want to re-try at increasingly longer durations for up to 24 hours, then implement a hard failure. Not only does this cover you for occasional glitches in the world wide interwebs, but will also cover you for any outages if your institution loses internet connection for a brief period, or if Canvas experiences either scheduled or unscheduled downtime (as two examples).
Now for the rest of error handling! Rest APIs employ standard HTTP error codes in the first instance, and then application specific error codes in the second instance. I highly recommend checking out HTTP Status Codes on Rest API Tutorial's site. You may not initially want to handle every single one individually, 200's indicate success (that's good), 400's and 500's indicate a problem (that's bad), now you know these exist you can handle them (that's good), having to handle them can be time consuming (that's bad). Kudos to anyone that can pick the pop culture reference/joke here!
I recommend looking at implementing an error handling module/subsystem/routine/procedure in your code, you write it once, can pass any results through it, and it will ensure consistent and robust error handling.
Finally, application errors! An application error is something that you try to do that Canvas itself says 'sorry, not going to happen'. This may be because you don't have permissions/access to execute your desired call, it may be because some of the required data is missing in your call, it may be because you have some flags incorrectly set, and so on. In these cases, it is essential to log what you tried to do, the calls you sent (ensure you do not record your API authentication token in logs), the result the API returned, and importantly the exact time and date (and timezone if relevant) you sent the call. Capturing all these details will make things far easier to diagnose if you need to contact Canvas support, and will greatly aid the team helping you to narrow down the problem.
While I have covered off most of this in the earlier sections, it is imperative to reiterate the concepts when discussing robust solutions. When you put together a script/program/integration that your institution comes to rely on for day-to-day operations, you should ensure you have accounted for these. The last thing you want to happen is an integration to work until you reach a level of scale where you start hitting the API limits, which could catch you off-guard.
Assuming you have already read the 'Understanding API rate-limits: a practical guide!' Section lets build on this further! One thing I did not touch on further is the detail on the best way to handle API rate limits. Rather than simply detecting, and backing off, you can go a step further, and your script/integration can be aware of the API limits.
If you have a read through Throttling - Canvas LMS REST API Documentation, it states: 'To assist applications with planning, every request will return a X-Request-Cost header that is a floating point number of the amount that request deducted from your remaining quota. If throttling is applicable to this request (it could be disabled on your Canvas installation, or you are whitelisted and not subject to throttling), there will also be a X-Rate-Limit-Remaining header of your remaining quota.'.
Therefore, if you are starting out, and wish to go down the path of parallelisation of your queries, I would strongly recommend you consider logging the X-Request-Cost in your debug logs, as well as the X-Rate-Limit-Remaining value. Knowledge is power, and ensuring you are aware of the maximum scale of your solution ensures you understands its limits.
You may find that you can run numbers (with some assumptions) and ascertain the acceptable throughput of your solution. If you find you are exhausting the API rate limit repeatedly, you may need to optimise your code (to reduce API calls if possible) or follow some of my earlier suggestions to use different API keys per call (as one example).
Things won't always be rosy and when you are programming that is OK, especially when starting out. For me, it is half the fun of learning a new system or programming language, as it helps to understand the inner workings. The below section covers some basic troubleshooting tips to help you out when things go wrong.
It may seem logical, but sometimes this will snag you, make sure you connected to the right environment. If you have multiple environments, this is an easy one to miss. Equally, once connected to the right environment, ensure your API token is valid and correct.
When you run into problems, first and foremost I would go to the Canvas Live API. The graphical interface is incredibly useful for walking you through API parameters, calls, and responses can be invaluable for helping to walk you through your logic. I know from personal experience that the Canvas Live API tool has helped me catch a few logic errors in calls that I was making (or expected returns).
The Canvas Developers group, along with the rest of the community holds a wealth of information and examples. When I started out, I spent a significant amount of time looking through examples, and hence I am an advocate of exploration.
If you haven't had any luck thus far, jump in to the Canvas Developers group where you can ask a question. There are many knowledgeable members of the community that may be able to help point you in the right direction.
What to include:
When you ask a question in the community related to APIs it is great to ensure you give as much information as possible. You can use this template to help you get started with the sorts of details that can help:
If your institution hosts Canvas with Instructure (Software as a Service), you can drop a request to Canvas Support. Also, if you are at the beginning of your implementation, your Canvas technical implementation staff member may be able to assist you (chat with your CSM). Please note that Canvas Support's responsibility is to help if the Canvas APIs are throwing application errors, if you are having issues with your particular language you are programming in, that is where you would need to fall back to the forums.
Should you contact Canvas Support, there are some things that you should include. The list is a copy of the list you would usually post to the Canvas Developers group, with some notable additions:
Providing support with all this information will let them check into the error you are receiving, and ensure that they have the right information at hand to assist you as best as possible.
Below is a list of specific cases that come up from time-to-time, these are things that seem to come up semi-regularly, therefore, I have added this section to cover them off and point you in the right direction.
When you are removing an enrolment, be sure you set the task parameter to deleted. If you do not pass the task parameter (or it has a syntax issue) the default behaviour of the API concludes the student enrolment. Check out the following question for some screenshots https://community.canvaslms.com/thread/23387-using-the-content-migration-api-for-blackboard-content#....
Most of these I have laced throughout this document, and are included here also as a complete list of resources:
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.