Update broke our links using http_referer

[lmd5adm]

We've had MANY links broken with this update.  We currently have a dynamic link that calls out to our server with a php program that looks at the HTTP_REFERER, grabs the course id, then translates that to our content space on our server.  We do it this way because we have so many sections that copy from a master section in to MANY sections and we do not want to force our design teams to have to edit each section in multiple places.  We need a solution to handle the scale. 

This solution has worked since our pilot last year.  Then we noticed that it broke on the mobile apps (but they were looking into it for us).  Now today is is broken across all of our online courses at PSU World Campus.

Anyone else rely on the http_referer or have a way to do links to external content areas so they can be scaled and dynamic?

[lmd5adm · Accepted solution]

<a class="Button--primary Button--large Button icon-calendar-month external" target="_blank" href="https://courses.worldcampus.psu.edu/lms/pageFromUrl2.php?request=/common/wrapsyllabus" title="" rel="noreferrer">

Notice the rel="noreferrer" attribute.  This is added by canvas via their react.js logic.  This was not that way before this weekend's production update.

We also found that we could modify these links to have a class of:  "not_external" and it will still open up in a new window but will permit the http_referer to be sent to our server and logic.  Changing many pages will be painful while courses are live.

[cesbrandt]

We have this exact same problem with our centralized content server. As an alternative to using anchors, we've looked at embedding the content, instead, but have found issues with some content types not rendering in browsers. This would also not help for files that we want the student to download.

An alternative that's being considered is to use a timed Event Listener to undo the addition of the relationship attribute in anchors with a reference to an external URL. We haven't reached a decision on it, because there's already several things we're doing this way and it can slow-down loading speeds.

If you'd like to give it a shot, here's the snippet:

// Filters anchors for external hyperlink references
function filterExternal(link) {
 var domain = new RegExp('/' + window.location.host + '/');
 return (!domain.test(link.href));
}
// Onload, search for anchors with relationship attribute and an external
//    hyperlink reference and remove the attribute
window.onload = function() {
 var timer = setInterval(function() {
  var links = document.getElementsByTagName('a'),
          j = 30,    // j = How long should it run for? (in seconds)
          k = 0.5;   // k = How often should it execute? (in seconds)
  for(var i = 0; i < links.length; i++) {
   if(links[i].hasAttribute('rel') && filterExternal(links[i])) {
    links[i].removeAttribute('rel');
   }
  }
  j = j - k;
  if(j <= 0) {
   clearInterval(timer);
  }
 }, (k * 1000)); // .5 second timeout
}

It hasn't been tested outside of a browser's Console, but testing via the Console and it performed as expected:

1. Start an interval of 0.5 seconds that will:
   1. Identify all anchors on the page; Set the length of time you want it to run the script; Set how often you want it to run
   2. Cycle through all the anchors and to remove the relationship attribute if both conditions are met:
      1. Anchor has the relationship attribute
      2. Anchor has an external hyperlink reference
   3. Update the timer to reflect the difference from the interval
   4. Clear the interval if the timer has run out

This is not a substitution for a fix from Instructure, merely a workaround that, hopefully, will work for you.

[lmd5adm]

We discovered that if we add the class "not_external" to the anchor tags we get the referrer. 

[cesbrandt]

That's great to hear! Combined with a target, which should already be present as proper form, this produces the appropriate functionality.

However, this particular issue would still affect Module links, though, as there is no option for setting classes or any flags, other than embed/new tab, when creating or modifying an external URL in the modules list. It would also require updating all the links in shells leading to external sources, which would be a not insignificant undertaking.

[lmd5adm]

The real solution would be for canvas to allow us to have some sort of setting that we can place on a course with a content url.  Then allow us to use a reserved variable in the link to build out the path to the content we need.  We had something like this in Angel called our $COURSES_DIR$ variable. 

Canvas allows us to have a setting where we put the url to the content such as http://courses.on.our.server/coursespace and then if we have an anchor with a class of "courses_dir" it builds that link for us on the click.  All that we may need to do is have the link point to the page on the server.  An example would be

<a class="courses_dir" href="content.html">Content Page From CMS</a>

yes, I'm going to write this up and submit it.  But I need some help getting it fine tuned and pushed up.

[cesbrandt]

If I follow you correctly, what you'll be looking to do is a cross-domain AJAX call, which is certainly possible in Canvas (I've done it to dynamically load the appropriate LiveChat support group), but I'm afraid my ability to aide with it would have to end at there.

Concerning the issue of the addition of the relationship attribute with a value of "noreferrer", we heard back from Instructure support that this change was made for security reasons and is being looked into to identify have an alternate solution could be implemented. If I hear anything else on it, I'll let you know.

[lmd5adm]

We are going to look at the possibility of finding a way to launch the LTI from inside of a canvas page.  If we do that, we capture the course in the LTI and eliminate the need for reauthenticating the user because we can trust the role of the launch.  If anyone has done this, please contact me.

[linkj]

Do you know if they might be working on a way to allow us to whitelist domains for referrer policy?

Like what drupal has? No referrer | Drupal.org 

As a multi-campus district we have multiple domains.

For locations using one domain "same-origin" would be strict enough

But at our location we would be cross domain to our district domain.

So one of the "origin..." or "strict-origin..." policies would have enabled our proxy link authentication.

[linkj]

Thanks!

class="not_external" worked for our EZProxy https links as well but that also turned off the default linking out to a new window/tab

So I had to add target="_blank" too

Although it does at rel="noopener noreferrer" automatically after adding target="_blank" , but it still opens and we get the referrer

[linkj]

Maybe unrelated, but is there a way add a HTLM5 meta tag to pages to allow https to http referral information to pass?

<meta name="referrer" content="always" />

Although I have a similar issue with all referer information being removed when trying to auto authenticate with EZproxy. Only embedded videos pass referring url information to our proxy server.

[cesbrandt]

I don't know much about proxy configurations, but in HTML5 the "always" value for the content attribute of a meta element is deprecated. While many browsers will recognize the value, it is discouraged that they be relied upon. Luckily, you could just add a second meta element with the updated value, that way you have the compatibility fallback in browsers that don't support the new values. The value you want is "unsafe-url". This will allow the full referrer URL (without any GET values) to be passed regardless of the security level.

HTML Standard

There are additional policy names that can be used that are not comparable to the deprecated values, should any of them better fit your needs: Referrer Policy

[linkj]

Thanks I'll ask our Canvas admin if he can add this to the Library external tools link in the side nav

We've been losing all our Google Analytics statistics for students coming to the library from Canvas