cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Community Advocate
Community Advocate

How to adapt to the undocumented JavaScript loading sequence changes?

Jump to solution

In the most recent Canvas Release Notes (2019-06-22) there was an undocumented change to what we believe is the loading sequence of JavaScript in Canvas. This has adversely affected anything we have done that changes the DOM. 

For example, one of our customisations is that we use the custom JavaScript to insert an RMIT branding image into every page in certain courses. We use customised JavaScript for this to ensure we can manage the branding from a central location (i.e in case colours or marketing changes etc.). 

As the loading sequence has changed, unless you now slow-load* the page, our JavaScript can't find the element that it needs to target because it doesn't appear yet. 

We are currently working on a temporary fix to ensure our assets are loading correctly - but would be interested to hear if anybody has advice on how we can meaningfully build custom JavaScript that will be robust enough to handle similar potential undocumented changes to Canvas? 

We understand customised JavaScript or CSS is not supported by Canvas, and it is clearly noted that there may be undocumented changes that occur - however, we are surprised that changes that would knowingly affect all 'Advanced CSS and JavaScript' users taking advantage of the open nature of Canvas were not communicated.

We are also very interested to understand WHY this change to the loading sequence was implemented as we simply don't understand...  

*In Google Chrome dev tools, the network panel allows you to slow load to the page using slow 3G under the throttling menu 

1 Solution

Accepted Solutions
Highlighted
Navigator

jayde.colquhoun@rmit.edu.au 

We're not loading any custom JavaScript right now, but I have had people say that QuizWiz stopped working for them and that $ (shortcut for jQuery) was unavailable.

I added a short custom JavaScript file to the theme on the beta instance so I could see what's happening.

One thing I notice when I watch the network traffic in Chrome is that they finally look like they got webpack working more like it's supposed to work. I'm getting approximately 38 really small scripts loaded instead of the one or two large ones that we used to get. Those scripts are loaded in the head element although they aren't there in the original HTML delivered from the server. It looks like Canvas is taking them from the body where there are link elements with rel="preload" and appending them to the head, which makes them execute. This means that the document is ready before the scripts are loaded and serves the purpose of making sure that the page isn't blocked because of script loading in the head. I suspect a reason for doing this is for browsers (Firefox) that don't support link with a rel=preload.

According to the Mozilla Developer Network (MDN), Preloading content with rel="preload" - HTML: Hypertext Markup Language | MDN 

The preload value of the <link> element's rel attribute lets you declare fetch requests in the HTML's <head>, specifying resources that your page will need very soon, which you want to start loading early in the page lifecycle, before browsers' main rendering machinery kicks in. This ensures they are available earlier and are less likely to block the page's render, improving performance.

Here's what I had in my custom JavaScript:

(function(){
console.log('Document state: ' + document.readyState);
const app = document.getElementById('application');
console.log(app);
})();‍‍‍‍‍

The document was not fully loaded when this executed. The state I got was interactive, not complete. According to MDN, Document.readyState - Web APIs | MDN 

The readyState of a document can be one of following:

  • loading: The document is still loading.
  • interactive: The document has finished loading and the document has been parsed but sub-resources such as images, stylesheets and frames are still loading.
  • complete: The document and all sub-resources have finished loading. The state indicates that the load event is about to fire.

Even though the document isn't fully loaded, the DOM is ready to be used. The div with id="application" was there. That means that anything present in the document itself is available for use, but things that get added afterwards through JavaScript (like the content on a content page) may not be.

None of that necessarily explains how to fix what is going on, but hopefully it sheds more detail on the loading order. In the past, it seems the custom JavaScript was loaded much later, but we haven't used any since Canvas added the dashboard sorting capability, and I don't remember for sure.

I have had to deal with content not being ready when your script executes, and it's becoming more prevalent, so let me tell you what not to do and how I fix it.

  • Don't delay for 500 ms or any other amount of time. This is very time sensitive and may work on one machine but not another. Sam is absolutely correct to reject that notion.
  • Some people like to use setTimeout and continually poll until things are ready. I remember Canvas itself doing that in pre-webpack days and thinking how bad of an idea that was (I'm not a JavaScript programmer by trade, so that assessment may have been in error). Make the timeout too low and it slows things down, make it too long and people can notice the lag.
  • I use Mutation Observers. This is event driven and happens as soon as the change to the DOM happens, rather than having to wait for the timeout to happen again. I've had to use mutation observers more and more as Canvas switches to ReactJS and InstUI. The content I need to modify may not be on the page yet and it may get refreshed and my changes wiped out. You can add a mutation observer and just let it listen trigger when the change happens.

I didn't know JavaScript until the last couple of years as I started writing the Canvancements, I had to teach it to myself and relied heavily on MDN and the "proper" way of doing things, not the old ways that people brought with them. This is where I got my distaste for setTimeout as it needlessly polls until something is available. Mutation Observers were the recommended way to go, so I learned how to use them.

Last week I wrote some quick code and without even checking to see if it was necessary, I included a mutation observer. The problem with mutation observers is that you have to attach them to something that is already in the document, meaning what is delivered as part of the HTML, if you want to be safe. That varies depending on whether it's a content page, an assignment, an index page, and so on. Most of the time id="application" is available, but depending on what you want to do, your stuff may not be in that div.

The code I wrote last week was to add a link to the Find a Question dialog. That dialog is ultimately attached to the body and so I had to observe the body to wait for it to appear. You want to find the closest parent that you know is going to be there and not be rewritten by React. Body is always there, but then it would get triggered every time a change was made. You can limit how far down those go, with childList or subtree in the init, but I encourage caution and not do too much or you're worse than setTimeout (childList isn't usually too bad, but a subtree on body would be).

Another issue with mutation observers is that they don't fire when there's not a mutation. For example, if you're looking for an item and it's already there, then it won't fire. What I do is check for the item I want. If it's not there, then I add a mutation observer that checks for it when fired, and if it is already there, then I just go ahead and move on.

Here's an example from the code I wrote last week (tweaked a little since I caught something as I wrote this)

function checkDialog(mutations, observer) {
const dialog = document.getElementById('find_question_dialog');
if (!dialog && typeof observer === 'undefined') {
const obs = new MutationObserver(checkDialog);
obs.observe(document.body, {
'childList' : true
});
}
if (dialog) {
if (typeof observer !== 'undefined') {
observer.disconnect();
}
addLink(dialog);
}
}‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

The first substantial line of my code calls checkDialog();

The first thing the checkDialog() function does is look for the element with id="find_question_dialog". This comes back null if it's not found, so !dialog means it wasn't found. With the first invocation, both mutations and observer are undefined, so it adds a mutation observer with an init of childList=true. Every time a direct child of the body is changed (inserted / deleted), then the mutation observer is called. I really don't need the first parameter mutations since all I need to do is check whether the item is present (the examples show iterating through the mutations, but that's not necessary). I do use observer so I can disconnect (stop listening) once the element is there.

Once the element is found, I'm ready to move on. First I check to see if the observer is undefined. It will be for the initial invocation, but if the code is fired because of a change to the children of the body, then observer is defined and there is a mutation observer, and I need to disconnect it. Then I go on to the next phase, which is the addLink() function.

As for what samuel.malcolm@rmit.edu.au wrote about wiki pages, Canvas has been injecting the content in via JavaScript for at least three years: Javascript only works in Pages under Course Navigation? 

Edit: I fixed some of the explanation on the link in the body vs script in the head. I was originally looking at the completed DOM rather than the way the file was delivered in HTML.

View solution in original post

20 Replies
Highlighted
Surveyor

Hi Jayde, I've been trying to listen for the load events for all the scripts in the DOM, the trouble is that we cant listen for the scripts already loaded when ours does and we cant know which ones we should be waiting for and which ones we should just assume as loaded.
Another suggested solution is to delay our script loading for an arbitrary length of time like 500ms, this is a bad idea as it will create an inconsistent experience for users with different types of internet connections. Which is especially pertinant in Australia.

Highlighted
Adventurer

Commenting to follow along as I am wondering if this is what may be causing some of my issues loading jQuery back into Canvas so that I can keep using the accordions from USU's Design Tool (formally known as Kennethware).

Highlighted

Hi Matthew, this would be the same issue im sure. The problem seems to be that even though the custom scirpt tags are defered, as in loaded after everything else, the Canvas JS isnt done injecting content into the custom content part of the wiki page when this occurs, meaning anything that targets content created by the user will break, including the accordions.

0 Kudos
Highlighted

At the moment, I have implemented a quick fix using setInterval to check if an element has loaded before running a script. Hope that this helps.

var search_timer = 0;
var search_element = setInterval(function(){
if ($('#element_id').length){
// clear interval
clearInterval(search_element);
// run custom script
init_custom_script();
}
// if element is not found in 2 seconds, stop interval
else if (search_timer === 20){
clearInterval(search_element);
}
else{
search_timer+=1;
}
},100);‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍
Highlighted
Navigator

jayde.colquhoun@rmit.edu.au 

We're not loading any custom JavaScript right now, but I have had people say that QuizWiz stopped working for them and that $ (shortcut for jQuery) was unavailable.

I added a short custom JavaScript file to the theme on the beta instance so I could see what's happening.

One thing I notice when I watch the network traffic in Chrome is that they finally look like they got webpack working more like it's supposed to work. I'm getting approximately 38 really small scripts loaded instead of the one or two large ones that we used to get. Those scripts are loaded in the head element although they aren't there in the original HTML delivered from the server. It looks like Canvas is taking them from the body where there are link elements with rel="preload" and appending them to the head, which makes them execute. This means that the document is ready before the scripts are loaded and serves the purpose of making sure that the page isn't blocked because of script loading in the head. I suspect a reason for doing this is for browsers (Firefox) that don't support link with a rel=preload.

According to the Mozilla Developer Network (MDN), Preloading content with rel="preload" - HTML: Hypertext Markup Language | MDN 

The preload value of the <link> element's rel attribute lets you declare fetch requests in the HTML's <head>, specifying resources that your page will need very soon, which you want to start loading early in the page lifecycle, before browsers' main rendering machinery kicks in. This ensures they are available earlier and are less likely to block the page's render, improving performance.

Here's what I had in my custom JavaScript:

(function(){
console.log('Document state: ' + document.readyState);
const app = document.getElementById('application');
console.log(app);
})();‍‍‍‍‍

The document was not fully loaded when this executed. The state I got was interactive, not complete. According to MDN, Document.readyState - Web APIs | MDN 

The readyState of a document can be one of following:

  • loading: The document is still loading.
  • interactive: The document has finished loading and the document has been parsed but sub-resources such as images, stylesheets and frames are still loading.
  • complete: The document and all sub-resources have finished loading. The state indicates that the load event is about to fire.

Even though the document isn't fully loaded, the DOM is ready to be used. The div with id="application" was there. That means that anything present in the document itself is available for use, but things that get added afterwards through JavaScript (like the content on a content page) may not be.

None of that necessarily explains how to fix what is going on, but hopefully it sheds more detail on the loading order. In the past, it seems the custom JavaScript was loaded much later, but we haven't used any since Canvas added the dashboard sorting capability, and I don't remember for sure.

I have had to deal with content not being ready when your script executes, and it's becoming more prevalent, so let me tell you what not to do and how I fix it.

  • Don't delay for 500 ms or any other amount of time. This is very time sensitive and may work on one machine but not another. Sam is absolutely correct to reject that notion.
  • Some people like to use setTimeout and continually poll until things are ready. I remember Canvas itself doing that in pre-webpack days and thinking how bad of an idea that was (I'm not a JavaScript programmer by trade, so that assessment may have been in error). Make the timeout too low and it slows things down, make it too long and people can notice the lag.
  • I use Mutation Observers. This is event driven and happens as soon as the change to the DOM happens, rather than having to wait for the timeout to happen again. I've had to use mutation observers more and more as Canvas switches to ReactJS and InstUI. The content I need to modify may not be on the page yet and it may get refreshed and my changes wiped out. You can add a mutation observer and just let it listen trigger when the change happens.

I didn't know JavaScript until the last couple of years as I started writing the Canvancements, I had to teach it to myself and relied heavily on MDN and the "proper" way of doing things, not the old ways that people brought with them. This is where I got my distaste for setTimeout as it needlessly polls until something is available. Mutation Observers were the recommended way to go, so I learned how to use them.

Last week I wrote some quick code and without even checking to see if it was necessary, I included a mutation observer. The problem with mutation observers is that you have to attach them to something that is already in the document, meaning what is delivered as part of the HTML, if you want to be safe. That varies depending on whether it's a content page, an assignment, an index page, and so on. Most of the time id="application" is available, but depending on what you want to do, your stuff may not be in that div.

The code I wrote last week was to add a link to the Find a Question dialog. That dialog is ultimately attached to the body and so I had to observe the body to wait for it to appear. You want to find the closest parent that you know is going to be there and not be rewritten by React. Body is always there, but then it would get triggered every time a change was made. You can limit how far down those go, with childList or subtree in the init, but I encourage caution and not do too much or you're worse than setTimeout (childList isn't usually too bad, but a subtree on body would be).

Another issue with mutation observers is that they don't fire when there's not a mutation. For example, if you're looking for an item and it's already there, then it won't fire. What I do is check for the item I want. If it's not there, then I add a mutation observer that checks for it when fired, and if it is already there, then I just go ahead and move on.

Here's an example from the code I wrote last week (tweaked a little since I caught something as I wrote this)

function checkDialog(mutations, observer) {
const dialog = document.getElementById('find_question_dialog');
if (!dialog && typeof observer === 'undefined') {
const obs = new MutationObserver(checkDialog);
obs.observe(document.body, {
'childList' : true
});
}
if (dialog) {
if (typeof observer !== 'undefined') {
observer.disconnect();
}
addLink(dialog);
}
}‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

The first substantial line of my code calls checkDialog();

The first thing the checkDialog() function does is look for the element with id="find_question_dialog". This comes back null if it's not found, so !dialog means it wasn't found. With the first invocation, both mutations and observer are undefined, so it adds a mutation observer with an init of childList=true. Every time a direct child of the body is changed (inserted / deleted), then the mutation observer is called. I really don't need the first parameter mutations since all I need to do is check whether the item is present (the examples show iterating through the mutations, but that's not necessary). I do use observer so I can disconnect (stop listening) once the element is there.

Once the element is found, I'm ready to move on. First I check to see if the observer is undefined. It will be for the initial invocation, but if the code is fired because of a change to the children of the body, then observer is defined and there is a mutation observer, and I need to disconnect it. Then I go on to the next phase, which is the addLink() function.

As for what samuel.malcolm@rmit.edu.au wrote about wiki pages, Canvas has been injecting the content in via JavaScript for at least three years: Javascript only works in Pages under Course Navigation? 

Edit: I fixed some of the explanation on the link in the body vs script in the head. I was originally looking at the completed DOM rather than the way the file was delivered in HTML.

View solution in original post

Highlighted

This is a really helpful and in-depth response. I think the mutation observer is exactly what we need and out of all the ideas ive heared today... theres been many... some my own... this seems to definitely be best practice of them.
Thank you for taking the time to author this, im going to try the mutation observer now

Highlighted

I think I need to clarify a little more because I think my issue may relate to this, but ultimately it is very different. I perhaps should just start a new thread. Basically I added custom JS to bring back the jQuery accordians. That code is actually still working well and I have no real issues build and using the accordions. However, the chunk of code is having adverse effects on other areas of Canvas (such as the SpeedGrader) I think due to this issue. Ultimately I think I need to pull the accordion code out to a separate file that is then only called from the main JS file when on content pages and the page editors.By that logic, the accordion code would not load in places that it is causing issues but still work where I need it to.

My biggest problem with this is I am not sure if I am correct or exactly how to write this.

0 Kudos
Highlighted

mjennings@uab.edu,

You can limit where content runs from within your main custom JavaScript without having to separate it out to another file. You can use something like this to make sure that you're on a page (of some kind). Of course, replace the console.log with the actual code that you want to run

if (/^\/courses\/\d+\/pages\//.test(window.location.pathname)) {
console.log('Running on a Page');
}

If you want it to run in other places, you could expand the pages to be (pages|assignments|discussions|quizzes) or whatever else you need it to match.

---

I'm not sure what you mean by the "page editors." Are you talking Rich Content Editor, pages / assignments / discussions, just pages and when you edit the page, or ???

Supplying working accordions inside the Rich Content Editor is going to be more difficult, and my third option is essentially the same thing.

---

When you installed your code to do accordions, how did you trigger it? Do you just look for .accordion or do you look for .enhanceable_content.accordion, or .user_content.accordion? 

I noticed last night when I was in the theme editor adding the custom JS that there was an Accordion section. I really hadn't paid much attention to it before, but it's marked up with a class of accordion and ui-accordion--mini.

If you're not being specific enough in your selector, you may be changing things that shouldn't be changed. This is what would happen if you were just using .accordion and not .enhanceable_content.accordion or .user_content.accordion.

A second possibility is that Canvas is picking up the .accordion and saying "let's act on this."

A third possibility is that the JS is fine, but you have an overly broad CSS selector that is modifying something. I don't see any accordions in SpeedGrader, but if you had them included in a discussion and viewed the discussion inside Speedgrader, it might show up (I'm scrunching my face and pulling my right shoulder up as I write that to indicate I really don't know).

I would start by checking any CSS you have and see what selectors are used. Then check the JS and see how your selecting what to operate on.

Finally, because it's more work but more likely to generate a solution, is to (in your beta instance) rename your trigger class from .accordion to something else like .uab_accordion. Reupload the JS and then go into SpeedGrader or another page where it's messing up. When I do this, I normally stick something like a console.log('Running accordion 1'); into the code. That lets me know it's running. The next time I update it, I would change the 1 to a 2. That way I know I have the latest version.

If SpeedGrader is working properly, then it might be the second possibility where Canvas is manipulating .accordion. You may need to go through and change all of your pages to use .uab_accordion. Canvas Data can help identify where those pages are if you have access to it.

If SpeedGrader works properly, then go into a page that has accordions and change the class there to be uab_accordion. Refresh the page and see if it works properly.

It's possible that what you're doing is related to what Canvas has done with the change in how the JS files are loaded. This would be the case where your code doesn't work even after changing the classname for the trigger. If it weren't for the weirdness in SpeedGrader, that might be my first thought.

Hopefully that gives you some things to think about.

Highlighted

james@richland.edu Thank You

THANK YOU! THANK YOU! THANK YOU!

Wrapping the code in that code to just run it on "Pages" did exactly what I needed. I'll be honest in that I read the first part tested it, sent it through to my supervisor and got approval for it to be loaded to fix the immediate issues we were experiencing. I just now read the full contents of your post. Unfortunately I probably am calling some broader items than I need to with some things, but that comes from not really know what I am doing along with mashing up different snippets of code from multiple people to solve different problems, each with their own writing style & preferences. 

I have this long term project on the board to start over & re-write/clean-up all of our customizations, but we are a staff member short and neck deep in about 5-8 other project that are demanding my attention. I am certain that you understand how that goes. Unfortunately, it will need to continue to hobble along in a piecemeal fashion for the time being.