Working with Refresh Tokens

[glparker]

Howdy,   

We are embarking on a new app which will be the first since the new requirement for using Refresh Tokens.      I'm confused by how the new workflow works, and am hopeful someone can help.

What is the process for getting the initial Access Token?   Has that changed?

Then, what is the process for getting a Refresh Token?   Is the Refresh Token different from the Access Token?   Or is it just the Access Token 'refreshed' with a new expiration time?  

Also, we had been storing Access Tokens in a database so as to not expire the tokens for our users and require reauthentication.   Should we now store the Refresh Token going fowrad, and use that to generate hour-long Access Tokens as needed?   Or is there another, better way to store 'something' and avoid our users needing to reauthenticate every time?   

Thanks, Glen

[muh]

You may refer to this document (OAuth2 - Canvas LMS REST API Documentation )

[glparker]

Thanks for the reply.   I've been pouring over that document, which is where my questions came from, as that document didn't answer them.  

That document would benefit from an example of what data we should expect to see returned in the Oauth call, what we would expect to see returned from the Refresh Token call, and best practices for dealing with/storing tokens .     

[themidiman]

I've not actually written code to do the oauth2 workflow, but I've worked with PHP code that does the oauth2 initial authentication and refresh workflow. It's the UDOIT project from University of Central Florida.

Here's their code repository. 

So the workflow is your application should as a first step get a new token using the oauth2 endpoint using a POST request with the request_type parameter set to 'authorization_code'. In the response from the ouath2 api endpoint, there should be a field called 'access_token' which is the token the app will use first. 

Your app should always be checking if the key has expired on the application landing page, and if so then make a new request to the oauth2 endpoint with a POST request with the rquest_type parapmeter set to 'refresh_token'

The code for the oauth2 steps are:

1. The index/landing page that should always attempt to authenticate via oauth2 and also see if a refresh token is needed
2. The code that establishes all ouath2 requests
3. The code that creates the POST requests of both types:
   

   $postdata = [
        'grant_type'    => 'authorization_code',
        'client_id'     => <your_application's_client_id>,
        'redirect_uri'  => <your_canvas_instance_redirect_uri>, //i.e. <instance>/login/oauth2/auth
        'client_secret' => <your_application_client_secret>,
        'code'          => <code_from_canvas_oauth_step1>
   ];
   return static::curl_oath_token(<your_canvas_instance_base_url>, $postdata);‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

   $postdata = [
        'grant_type'    => 'refresh_token',
        'client_id'     => <your_application's_client_id>,
        'redirect_uri'  => <your_canvas_instance_redirect_uri>, //i.e. <instance>/login/oauth2/auth
        'client_secret' => <your_application's_client_secret>,     
       'refresh_token' => <refresh_token_from_initial_access_token_request>
   ];
   $json_result = static::curl_oath_token(<your_canvas_instance_base_url>, $postdata);
   return isset($json_result->access_token) ? $json_result->access_token : false;‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

   
   Hope that helps

[jpoulos]

 @glparker ‌

I believe the information you are asking for is already in the documentation, but I'm always open to suggestions on how to make it more clear.

To see examples of the response you can expect when exchanging your code for token, this part of the documentation should help: OAuth2 Endpoints - Canvas LMS REST API Documentation 

In terms of best practices for storing tokens, we do attempt that here: OAuth2 - Canvas LMS REST API Documentation 

Again, I'm open to recommendations one specific improvements you'd like to see with the documentation, so please be critical

Jesse Poulos

Partner Integrations Specialist

Instructure

[glparker]

Hi Jesse,

Thanks for the update.   That API doc on Endpoints is helpful.

My confusion still stems from storing and dealing with the Two tokens.   

Storage:  

Do you recommend we store both the Refresh and Access token in our database, or just the Refresh, which we can use to generate a valid Access token for each API call?

Do you recommend we store the expire timestamp or the Access Token?   

Do you have general guidelines for what data elements a Token storage system might include?

Dealing with Expiration:  

Should we compute when the token will expire and preemptively generate a Refresh request for an updated Access Token, or is it better to make the call using the expired Access Token, receive the 401 Unauthorized error, THEN generate the Refresh request, finally make the API call again with the refreshed Access Token?

(Said another way, is it better to preempt the 401 error or to react to the 401?)

Finally, do you have some psuedo code for what the entire workflow might look like for making subsequent API calls that accounts for potentially expired Access Tokens?   I think you explained the process of getting the initial token(s) well, but the process for using refresh tokens was fuzzy.

Thanks, Glen

[justinball]

Hi Glen,

We recently had to make updates to our software to handle the refresh token as well. Here's a bit of example code in Ruby: lms_api/canvas.rb at master · atomicjolt/lms_api · GitHub 

In Particular look at the refreshably and refresh_token methods. In our code we make the request to Canvas. We check all responses for the http status code. If we get a 401 and the "www-authenticate" header is Bearer realm="canvas-lms" then we know that our token has expired and we throw a LMS::Canvas::RefreshTokenRequired exception. We catch that exception and lock the record in our postgres db. That way only one process will attempt to update the refresh token. We then make a request to update the token.

Once we have a new refresh token we update and unlock the record in the database.

Here's a video where I walk through the process:

Ruby Ep. 7: Concurrency and avoiding race conditions with Ruby on Rails - YouTube 

We've since refactored our code so it doesn't exactly match the links below, but hopefully it helps.

If you're using Ruby we have a starter apps we use when building LTI tools / Canvas API integrations. The code is MIT licensed. If it helps feel free to use it or extract code from it:

GitHub - atomicjolt/lti_starter_app 

The code that handles the refresh token is in the lms_api Ruby Gem we built:

GitHub - atomicjolt/lms_api: Ruby wrapper for the Canvas API 

Thanks,

Justin

[jpoulos · Accepted solution]

Thanks  @justinball ‌, this a great info for the community and should help with the ask for code examples.

Just to clarify, in case it wasn't already, the refresh token will never change unless you run through the entire OAuth2 process again.

Regarding storage, you at least have to store the user's token temporarily in order to use it, you can store it in a permanent DB as well. The refresh token should be stored permanantly; Just to clarify, in case it wasn't already, the refresh token will never change unless you run through the entire OAuth2 process again, so you can keep re-using it.

Storing the expiration time isn't really necessary if you take the responsive approach that Justin has discussed. Personally, I prefer to store the expiration time of the token. To do this, when I get the token, I just set a field "expires_at" to the current time + 1 hour. During each API call. I then run a pre-check to determine wether the current time is exceeded, and if so, I run the refresh process and update the stored API token in the DB before running the API request.

Both ways work (pro-active and re-active), it's not clear to me whether one is better than the other in terms of system performance. I imagine checking current time against the expiration time for every request could add up to more processing power for your servers, but I'll leave that to the computer scientists to determine where or if that starts to cause issues.

I personally like to store the API token in my User table with the following columns:

id: this is my own internal ID

primary_email: 
canvas_user_id:  the user id in the canvas url

user_id: the opaque user id sent during launch,

canvas_api_token: the token obtained during either the first OAuth2 flow or the refresh flow

canvas_api_refresh_token: the refresh token obtained during the last complete OAuth2 flow

token_expires_at: the time at which the token expires

When the app launches, I look up the canvas_api_token based on the user_id field since this never changes in Canvas.