cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
lbeveridge
Community Participant

422 Error when making POST Request to Favorite Courses API

Jump to solution

Hello,

I am working some custom Javascript that needs to be able to add and remove favorite courses for users through the Canvas API. I can get the list of a user's favorites without any issues, but I am getting a 422 "unprocessable entity" response when I try to do a POST request to add a favorite course for a user.

First, I tried making the POST request without any authorization, since the GET request for favorite courses does not require it, however I got the 422 response. Then, I used Chrome's dev tools to look into the request that Canvas sends when a user clicks on the start icon in the "All Courses" menu to favorite a course. I found that the payload Canvas sends includes an "authenticity_token" which is called the "_csrf_token" in Canvas's cookies. This token is regenerated each time it is used, but I tried manually adding it to my code to see if that would solve the error. Here is the code I am currently using:

 

data = {}
fetch('<our canvas url>/api/v1/users/self/favorites/courses/<course number>', {
  method: 'POST',
  headers: {'Content-Length': 0, authenticity_token: '<csrf_token copied from Canvas's cookies>'},
  body: JSON.stringify(data),
})
.then(response => response.json())
.then(data => {
  console.log('Success:', data);
})
.catch((error) => {
  console.error('Error:', error);
});

 

With this code, I still unfortunately get the 422 response. The user I am testing this with is enrolled in the course that I am trying to favorite, and the course is not already favorited. I am thinking I must be doing something wrong with this request!

I would greatly appreciate assistance with this if anyone has ideas as to why this error is occurring!

Labels (5)
0 Kudos
1 Solution

Accepted Solutions
robotcars
Community Champion

@lbeveridge 

I've never tried to pass the token in the form data, just the headers. I did some practice with this awhile back for XHR testing and applied it to your fetch. I was able to make this work. Since the token is refreshed, I use a function to strip the token from the cookie and decode it, the function making it reusable for subsequent requests. You can see this by making any AJAX request on the page and calling CSRFtoken().

 

const CSRFtoken = function() {
  return decodeURIComponent((document.cookie.match('(^|;) *_csrf_token=([^;]*)') || '')[2])
}

fetch('/api/v1/users/self/favorites/courses/1234567', {
    method: 'POST',
    headers: {
      'content-type': 'application/json',
      'accept': 'application/json',
      'X-CSRF-Token': CSRFtoken()
    },
    //body: JSON.stringify(data),
  })
  .then(response => response.json())
  .then(data => {
    console.log('Success:', data);
  })
  .catch((error) => {
    console.error('Error:', error);
  });

 

 

 

 

View solution in original post

13 Replies
James
Community Champion

@lbeveridge 

You don't send the CSRF token as an authenticity header. You send it as part of the data that becomes the body rather than as a header. Move authenticity_token from the header object to the data object.

Since you're sending JSON, you may need to include "content-type: application/json" in the headers. You generally do not need to include content-length. I tend to throw in an "accept: application/json" header as well.

I didn't check the rest of the code for any problems, just the thing that jumps out at me. I've used the CSRF token before for API calls, so I just searched my GitHub repository to see how I handled it. I was looking at a PUT instead of a POST, but that shouldn't affect where the authenticity_token goes.

lbeveridge
Community Participant

Thank you for the help here James! I appreciate it!

I went ahead and tried adding those fields in, and moving the CSRF token into the body, but I'm still getting the same error unfortunately. Here is the code I have now:

data = {'authenticity_token': '<CSRF Token>'}
fetch('<Our Canvas Domain>/api/v1/users/self/favorites/courses/656', {
  method: 'POST',
  headers: {'content-type': 'application/json', 'accept': 'application/json'},
  body: JSON.stringify(data),
})
.then(response => response.json())
.then(data => {
  console.log('Success:', data);
})
.catch((error) => {
  console.error('Error:', error);
});

If you have any other ideas about what mistake I may be making, I would be grateful for the help. Thank you!

robotcars
Community Champion

@lbeveridge 

I've never tried to pass the token in the form data, just the headers. I did some practice with this awhile back for XHR testing and applied it to your fetch. I was able to make this work. Since the token is refreshed, I use a function to strip the token from the cookie and decode it, the function making it reusable for subsequent requests. You can see this by making any AJAX request on the page and calling CSRFtoken().

 

const CSRFtoken = function() {
  return decodeURIComponent((document.cookie.match('(^|;) *_csrf_token=([^;]*)') || '')[2])
}

fetch('/api/v1/users/self/favorites/courses/1234567', {
    method: 'POST',
    headers: {
      'content-type': 'application/json',
      'accept': 'application/json',
      'X-CSRF-Token': CSRFtoken()
    },
    //body: JSON.stringify(data),
  })
  .then(response => response.json())
  .then(data => {
    console.log('Success:', data);
  })
  .catch((error) => {
    console.error('Error:', error);
  });

 

 

 

 

View solution in original post

James
Community Champion

@lbeveridge 

Not necessarily relevant to the issue, but are you trying to add this to the custom global JavaScript that runs inside Canvas or are you trying to do this from outside Canvas?

An example of the first kind would be to go through and make something a favorite every time people log into Canvas (a course that cannot be un-favorited). An example of the second kind would be if you want to go through and add a favorite to everyone and then be done with it and they can remove it if they want?

Both can be done with JavaScript. The first kind needs the CSRF token and runs within the browser, but may not get those people coming in through the mobile apps. The second kinds needs to be ran externally, say using a NodeJS script but needs to use the authorization header and won't have access to a CSRF token. If you're trying to take the CSRF token from within Canvas and then use it externally, you're going to have issues (401 error invalid access token).

The reason I ask is because if you want to do it outside of the browser, you can do it for all users using masquerading and an access token belonging to someone with the ability to masquerade. If I was a student and someone kept on favoriting a course for me each time I logged in and I kept on unfavoriting it, I wouldn't be happy as space on the dashboard is limited.

Okay, now to the issue. I'm including the whole investigative process so other people with the issue can come along later and have a resource. What I think the issue is at the bottom, but no one has ever accused me of being a TL;DR kind of person.

There are a couple of things that come to mind.

The first was is the person who is running the script (the person you're logged in as within Canvas or the person owning the token) actually enrolled in course 656? That is, can you go to the Courses page for that user and click the favorites to add or remove it? Not all courses are can be made favorites. However, I don't think this is it. Canvas returns a 200 OK when I try to add a favorite for a person who isn't enrolled in a course (maybe they will be at some time in the future).

The second is whether the course ID is correct. If I try this with a bogus ID that doesn't belong to us, I get a 404 Not Found error, not a 422 code.

So the next thing I do is to open up the developer tools in Chrome and look at the network traffic that Canvas sends to see what's missing. The only two things it's sending in the payload is the _method and the authenticity token. I notice you don't have _method in there.

So, I right click on the request and copy it as a fetch so I can replay it without the _method. Nope, that still works.

Then I look at the headers and something jumps out at me. They do include the CSRF token in the header, but they don't call it authenticity_token and then don't call it authorization. It's called x-csrf-token. So I take out that header and it still works.

That makes me wonder what you are including in the authenticity_token. It should just be the value of the CSRF.

If I remove the x-csrf-token header and garble the what I'm sending as an authenticity_token, I'm able to get the 422 error.

Garbling the authenticity_token so that it doesn't match the x-csrf-token gives a 500 code.

Changing the x-csrf-token while using the right authenticity_token works.

It seems that the x-csrf-token header is optional. If it is included, you do not need to include the authenticity_token in the data, but if you do include the authenticity_token in the data, then the x-csrf-token is overridden by the authenticity_token.

That seems that the easiest way to do it would be to include the x-csrf-token in the header and not mess with a body at all.

But all that then comes down to what you are using as the token. You didn't share that (good reason for not doing so), but then you have to wonder, after tracking everything else down, whether you're sending the right thing or not.

You should send just the value of the csrf-token and not the _crsf_token= portion. Depending on where you send it will determine whether you should decode it or not first.

If I look at my set-cookie header that is returned as part of the response or I use document.cookie to obtain the _csrf_token value, I get a uriEncoded value that contains things like %2F, %2B, and %3D.

Those cannot be passed in either the x-csrf-token or authenticity_token (if you are using JSON). You have to pass the decoded form.

If I let run my cookie _csrf_token through decodeURIComponent(), then I can send it as either x-csrf-token or authenticity_token. Note that decodeURI will not fix it, it needs to be decodeURIComponent.

If you are using the content-type of application/x-www-form-urlencoded; charset=UTF-8, then you must include the encoded form. If you try to send the decoded form as the authenticity_token, it will give a 422 error.

 

In short, use a header of x-csrf-token and make sure you pass the _csrf_token cookie value through decodeURIComponent() before doing so.

James
Community Champion

Nice job Robert ( @robotcars ). Reminds me of the good old days when there was no response when I started typing and there was by the time I was done and hit post.  You beat me by almost 30 minutes, which means I spent way too much time on it.

@lbeveridge - see what Robert said. We said the same thing, he just did it much more succinctly.

robotcars
Community Champion

Those were good days. They will come back. I thought you were gonna beat me to it. Your detailed explanations are always on point!

 

 

James
Community Champion

@robotcars 

Speaking of old times ... something bothered me about the regexp when I first saw it. It works, but it took a while to figure out why it worked. This is me mostly rambling ... avoiding grading finals.

My first question was whether the || '' was misplaced? I originally though you were trying to return '' for an undefined value, but then it hit me that you were trying to avoid the error of when the match comes back null.

document.cookie.match('(^|;) *_csrf_token=([^;]*)') || '')[2]

 

The regexp.match() returns null, the same as regexp.exec(), when the string is not found. If _csrf_token is not in the cookies, then the match returns null and you assign it the empty string '', which is a string and not an array. I originally thought that accessing [2] on a string would return an error, but it turns out it returns the character in that 0-indexed position. The empty string has 0 length, so accessing ''[2] returns undefined and decodeURIComponent(undefined)='undefined' (it literally returns the string 'undefined').

How bad is it that I originally ended that sentence with a semicolon?

Since I had always used substr() to get a portion of a string, I didn't know the other method worked, so I'm glad I dug into it.

Since match returns an array, my internal linter (under heavy influence from the external linters) would suggest that I'm returning different types and proffer that I should use || [] instead of || '', but the end result is the same. Rather than getting undefined because the length of the string is 0, you're getting undefined because the length of the array is 0.

The external linter I'm using keeps telling me to reduce the complexity of my code from 98 to 15 or less. It really doesn't like deeply-nested if statements within loops.

Now that I know your code is good and works, the first thing I saw was that you could use a non-capturing group ?: as in (?:^|;) to use [1] at the end instead of [2].

I played around with it some more. I investigated word boundaries \b, but it turned out that won't work since it picks up on things like !_csrf_token.

What you're doing is simpler than what I came around to. I used regexp early on, especially with the link headers for pagination, but then switched to splitting the cookies into an array and comparing one by one.

The JavaScript Info website says the easiest way to get a cookie is through a regular expression. Then they show how to creating a regular expression from the name instead, which is essentially what you've done. They do use (?:^|; ) -- note there's a space after the semicolon, so it looks like the space is required after a semicolon to be a valid cookie. I like your approach of making it optional instead. They also split up the thing into two lines. One that does the match and then one that checks to see if there was a match before decoding it or returning undefined (not the string) if it isn't there. That is a cleaner way than using the || '' or || [] and decoding an undefined value. Of course, given the system we're operating in, we should always have the _csrf_token header available to us so we take shortcuts that wouldn't work other places.

You know that I don't like doing things the established way and I know you like writing one-liners 🙂 Here's my most recent attempt [at least 3 years ago] at getting the csrf token cookie. I used a generic getCookie() rather than writing it specifically for the csrf token -- even though the only time I ever called it was for _csrf_token.

  function getCookie(name) {
    return document.cookie.split(';').reduce((a, c) => {
      const d = c.trim().split('=', 2);
      return d[0] === name ? decodeURIComponent(d[1]) : a;
    }, '');
  }

 

It accomplishes the same thing as the getCookie() function from JavaScript Info, but it returns an empty string instead of 'undefined' when it cannot find the cookie. 'undefined' is truthy while '' is not, so if you wanted to check to see if a cookie was present before adding it unconditionally, then you might run into issues. If I wanted to do that, I should probably start with null or undefined instead of '' in my reduce function. Like you, I added the results unconditionally, so I didn't want undefined.

The file that had that code in it was the one I was looking at when I made my original response to this thread. I did put it into the data.authenticity_token since that's what Canvas did and so when I looked at the original code, I knew that I had done something like that before. It was so similar to what I had that I skipped past the part about getting the token and assumed it was correct. That code also included bottleneck so I made sure I didn't hit the API too heavily (it turns out there are too many missing labels that needed fixed to send them all at once).

MattHanes
Community Champion

@James wrote:

How bad is it that I originally ended that sentence with a semicolon?


I'm just here to say that I read this and quite literally laughed out loud and snorted my diet root beer up my nose.

lbeveridge
Community Participant

@robotcars & @James Thank you both for your help with this, it is working now!

My issue seems to have been a combination of me calling the CSRF token 'authenticity_token' instead of 'X-CSRF-Token' and the way I was manually copying in the CSRF token. I had been copy pasting the CSRF token out of the cookies each time it updated without also selecting "show URL decoded" for the cookie value. The un-decoded value causes the request to fail, whereas if I paste in the decoded value, it works. Either way, having the function to automatically read in the CSRF token, as well as changing the value in the header to 'X-CSRF-Token' solves it!

Thank you much for the assistance with this!