Is there a way to prevent a course from being indexed if it's set to public?
There is via the robots.txt file, but that is only editable by Instructure. If you read the current file, it only excludes /page_views/, which doesn't exist in Canvas. The comments in the file (which are pretty standard) state to ban all spiders from the site, uncomment the next 2 lines. But the next 2 lines only block all spiders from /page_views/. No matter what, though, that file is not right as it is disallowing crawling on a page that doesn't exist.
Further, the robots.txt file is the same on canvas hosting as it is in github here:
canvas-lms/robots.txt at master · instructure/canvas-lms · GitHub
which hasn't been edited since 2011 - making me think it's been forgotten about; That the page it does block existed in Canvas at one point.
I opened a ticket with support quite a while ago (over a year?) and even had a meeting with our CSM and someone on the development/engineering side. The end response was they would update the file - but it hasn't happened yet. The support case is still open.
If you would like the case number, let me know.
Retrieving data ...