Http request error

[nschutz]

Hello,

I am new to Canvas and trying to get started on using the APIs. When I put a simple request in my browser, https://domain.instructure.com/api/v1/courses/ , I get the following error displayed.

SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data

Could it be I am an admin and not enrolled in any courses?

I get the same error when I send this request, https://domain.instructure.com/api/v1/courses/$courseid/pages 

even though I have access to this course.

I appreciate your insights.

[James]

 @nschutz ,

You are getting a list of courses, but Canvas puts a while(1); in the front of it since your browser hasn't told it that it can handle JSON results. The while(1); is not valid JSON and so the browser is complaining.

Here's what I get when I try the first request inside Chrome.

Here's what I get when I try it in Firefox

Okay, that's not technically true. What I get is this:

Recent versions of Firefox handle JSON and allow you to look at it in formatted JSON view or in Raw Data mode.

But ... but ... but ... I'm using Firefox, you say.

Okay, so what you're seeing is this error:

If you switch to Raw Data, you'll see the while(1); at the beginning as I showed before.

The problem is that Firefox does not accept JSON responses by default and you need to tell it to do that if you want to do this kind of thing in your browser. 

This is controlled by the network.http.accept.default setting. By default, it is set to 

text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Since that doesn't contain application/json, Canvas puts the while(1); in front.

What I've done is modified mine to include the application/json. Here's what mine looks like.

text/html,application/xhtml+xml,application/xml,application/json;q=0.9,*/*;q=0.8

You might be able to list the json before the xml, but I didn't want to risk breaking something and so I just added it to the end.

Now that I've explained the problem, the next question is how to fix it.

1. Type about:config in the location (URL) bar.
   
2. Accept the risk of voiding your warranty
   
3. Search for network.http.accept.default in the search box
   
4. Right click the mouse and choose Modify
   
   
5. Add ,application/json between the application/xml and the ;q=0.9
   
   
6. Click OK

After going through that process, you can now refresh your page from Canvas and see a list of the courses nicely formatted. Once you start telling websites that you understand application/json, then Canvas stops putting the while(1); in there and it is recognizable JSON.

If you ever need to set it back to the default -- say because you want to show someone the steps of how to set it up like I did tonight, then click Reset in step 4.

[nschutz]

Hi James,

Thank you so much for the quick replay and detailed explanations. Really really helpful! I did a few hours of reading discussion groups and tutorials on how to use the API last night, and it appears that I can run a web app (on localhost) with PHP and test the API code this way? Does it seem like a good starting place? Do you have other recommendations? The API code will be used mainly to bulk update our course content.

[James]

I suppose you could run a web app on localhost, but there are probably easier ways. I use a mix of PHP or PERL to interact with the API, but that's me writing code, not running an app.

For specific testing of the API, I would probably recommend getting a ReST client for your browser. Chrome has Advanced ReST client (ARC) that I use. Here in the Community, it seems that Postman gets the most mention and few use ARC. There's also the Live API, which is to take your main Canvas Instance and tack /doc/api/live onto the end. It requires an access token, but the instructions are right there with the form and then you can look at how it formats things to see how it should be done.

[nschutz]

Hi James,

Your suggestions have been tremendous! It appears that I can't do bulk updates with Postman, only one call at a time. Then I came across your document Adjust All Assignment Dates on One Page and it seems to be closer to what I am trying to do even though this is a spreadsheet that only adjust certain fields, not the fields I need to change.I need to bulk update some fields in our quizzes. I assume I just need to go to the Script editor and make changes accordingly? Are there other ways to do bulk updates? There is also a twist to the bulk update as it can only do the update when a condition is met.

[James]

Depending on what you need to do with them, quizzes can be a beast. The quiz settings are fairly straight forward and could be adapted to the spreadsheet. The quiz questions are extremely complex and depend on the type of question.

You've also got to watch things on the quizzes because what you download cannot be turned around and sent back directly to Canvas in an update. The updates and create statements require everything be under a 'quiz' property, but they don't come that way from Canvas during a list / get statement.

I used some PHP code at the beginning of the fall semester to go through and change all of my quizzes to use the average grade rather than the highest grade. I've got a library I wrote to handle the API calls, but it probably would have been easier if I would have just found a REST client for PHP and used it. Guzzle has some potential and allows for calls to be made in parallel, but it would mean rewriting a bunch of legacy code and it's not high on my list of things to do since it works as is.

If PHP isn't your thing, there are REST libraries for most popular programming languages. I wouldn't use the spreadsheet if the process is automatic as it's one of the slower interfaces. The spreadsheet worked well because there was lots of manual changes to the data and it fit with the tabular data. But if your updates are bulk but not automatic, then it might work for you.

[nschutz]

Thanks for the helpful warnings. They are bulk updates but not automatic. I don't need to update quiz questions, just settings, fortunately, but they are conditional updates meaning only records that meet certain criteria can be updated. Will the spreadsheet approach be able to handle conditions?

You mentioned using PHP in the beginning of fall, what client did you use to make the API calls? I will also look into Guzzle. Thanks for sharing your wealth of experience!

[James]

I ended up going with the built-in cURL libraries with PHP and writing my own interface to handle things like pagination. If there is something out there that will do that already, it's definitely easier. I've always been one to keep reinventing the wheel. It helps me re-examine old practices, see if there's a better way, make sure I understand things. Not every one has that time or desire.

The spreadsheet approach could handle things, but there may be easier ways. The code that I wrote to change the highest to average checked to make sure that it was a quiz (not a survey), had multiple attempts, and was set to highest. So you can program in the logic.

The spreadsheet approach would be good for a generic quiz setting editor. You could show all of the quiz settings for all of the quizzes and then make updates to them all in one spot. That's not a bad idea, just not the reason why I wrote the original spreadsheet.

[nschutz]

I made a copy of dueDates.gs in the spreadsheet and modified the copy to make other quiz settings change. It worked great! Thanks again.

The part I haven't figured out is when I go to the Canvas menu on the sheet and select a menu item, like Load Due Dates, how does the spreadsheet know which gs file to execute, the original or the copy because both script file has the same function names, listDueDates or setDueDates etc.? To be safe, I executed my copy from the backend, not thru the menu.

[James]

Wow, that was fast. Glad you were able to get it going so quickly. It looks for the name of the function and it doesn't matter which GS file it's in. Now, if you had the same function in multiple files, I'm not sure. I don't know if it's whichever one comes last in the load process or if it looks for one in the same file as the calling function. The best thing to do would be to rename all of my functions or delete my GS file once yours is working and you don't need mine anymore. Other possibilities would be to make the first line of my functions just be return(); so that they don't do anything, or comment them out.

To answer, the question, I created three files. The PRIMARY file contained the onOpen() function that loaded the menu and a logSomething() function. The logSomething() function was repeated in a SECONDARY file. There was a logSomethingElse() in the SECONDARY and TERTIARY files, but not the PRIMARY one.

/**
* @OnlyCurrentDoc
*/

// PRIMARY FILE

function onOpen() {
  var dt = new Date();
  var ui = SpreadsheetApp.getUi();
  ui.createMenu('Canvas')
  .addItem('Something', 'logSomething')
  .addItem('Something Else', 'logSomethingElse')
  .addToUi();
  return;
}

function logSomething() {
  Logger.log('SOMETHING - PRIMARY');
}

// SECONDARY FILE

function logSomething() {
  Logger.log('SOMETHING - SECONDARY FILE');
}

function logSomethingElse() {
  Logger.log('SOMETHING ELSE - SECONDARY');
}

// TERTIARY FILE

function logSomethingElse() {
  Logger.log('SOMETHING ELSE - TERTIARY');
}
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Choosing Something from the menu gives the Logger.log() statement from the SECONDARY file and choosing Something Else from the menu gives the Logger.log() statement from the TERTIARY file.

That is, it appears to give the last one that was loaded. How does it determine last loaded. It looks like the order the scripts were created in. Even if you go to View > Alphabetical in the Script Editor and reload, it still puts TERTIARY when I run the Something Else function.

On a whim, I tried changing the order I saved the files in and that didn't seem to make a difference either.

So, it sounds like if you copied mine and then changed them, it would run yours. However, I would not trust that to be deterministic unless you found it written down somewhere by someone who knows. It's best to not reuse a function name.

[nschutz]

It makes total sense. I renamed the function names in your original gs file that are duplicated in my file.

[chriscas]

Hi  @James ‌,

Is this Firefox JSON trick still working for you?  I used your instructions to get the JSON viewer working in Firefox for the sis_imports page, and it worked beautifully for quite a while.  When I went to the page this week, it's back to showing the while(1); in front of the JSON.  I'm not sure if something changed in Firefox or Canvas to cause this, but I miss the pretty formatted JSON view, and I'd love to get it back again (without having to copy/paste into other apps or use add-ons).

-Chris

[julian_ebeli]

Hi Chris,

I'm using the JSON lite firefox plugin.

It renders and formats the json ignoring 'while(1)'

[julian_ebeli]

Having just re-imaged my machine, I needed to reinstall json lite. I wasn't working at first because you have to turn off firefox's own json render.

GitHub - lauriro/json-lite: Fast non-blocking JSON viewer for Chrome and Firefox has the instructions

Go to about:config and set 'devtools.jsonview.enabled' to false

Now it works!

[chriscas]

Thank you,  @julian_ebeli ‌!

That was the setting change I'd been apparently missing for a few months to actually get JSON Lite working.  I had given up on the entire thing a while ago, but after disabling that setting, everything is working as I needed now.

-Chris

[James]

Chris,

It is not working any more for me. I hadn't tested it recently until just now.

When I look the request header sent to the API from Firefox, it has 

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

It stripped out the application/json part. At first, I thought maybe it got reset during an upgrade.

I went into about:config and my network.http.accept.default is set to text/html,application/xhtml+xml,application/xml,application/json;q=0.9,*/*;q

So I did a little looking and found the List of default Accept values on MDN and found that since Firefox 66, it is text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8.

That isn't so bad, it's just the default. But it's the comment next to it that was unpleasant. "In Firefox 65 and earlier, this value can be modified using the network.http.accept.default parameter." Firefox 66 was released on March 19, 2019. It seems that we cannot use that configuration option (it doesn't even seem to exist any more)

I haven't looked at the JSON Lite that  @julian_ebeli  mentioned. It looks pretty popular and has a high rating.

Since JSON Lite looked like it repeated functionality that was already in Firefox for displaying JSON, I decided to look into what could be done to make sure Canvas didn't send the while(1); at all. I downloaded and test Header Editor and added a rule. It works, but it seems limited in that each rule can only have one header that is modified. I would need to add a rule if I also wanted to specify the content-type. It does support regular expressions for the URL, so I could make it match the production, test, and beta instance with one rule.

I'm not recommending Header Editor, just saying I checked it out as an alternative to what used to be part of Firefox.

[julian_ebeli]

Hi

Use Postman | API Development Environment to experiment with api calls.

You will need API keys. You can generate them in your profile settings.

Eventually you will move on from postman and integrate the calls you make into a more involved webapp of some kind. You need some sort of script to deal with the return data.

I use python with the requests library to do app prototyping.

Good luck