Prevent indexing?

[abunag]

Is there a way to prevent a course from being indexed if it's set to public?

[ahess4]

There is via the robots.txt file, but that is only editable by Instructure. If you read the current file, it only excludes /page_views/, which doesn't exist in Canvas. The comments in the file (which are pretty standard) state to ban all spiders from the site, uncomment the next 2 lines. But the next 2 lines only block all spiders from /page_views/. No matter what, though, that file is not right as it is disallowing crawling on a page that doesn't exist.

Further, the robots.txt file is the same on canvas hosting as it is in github here: 

canvas-lms/robots.txt at master · instructure/canvas-lms · GitHub 

which hasn't been edited since 2011 - making me think it's been forgotten about; That the page it does block existed in Canvas at one point.

I opened a ticket with support quite a while ago (over a year?) and even had a meeting with our CSM and someone on the development/engineering side. The end response was they would update the file - but it hasn't happened yet. The support case is still open.

If you would like the case number, let me know.

Thanks,

Anthony Hess